AI and Crypto: Countering Centralization | Casey Caruso & Daniel Shorr

Transcript

so AI is one of the most transformative Technologies we've ever made ever and I've known that for a long time since I've been a kid it's an inherently centralizing technology and the reason it is if it's not clear is because you are more efficient when you have gpus connected collocated you can do more from a fundamental Hardware level and so this has this huge ripple effect of basically centralizing the entire thing as a believer of decentralized Technology I've always felt that one of the biggest threats we have at honestly not to sounds super philosophical but Humanity faces right now is centralizing AI to a

point that the Innovation doesn't compound all right everyone so on Empire you obviously know that we talk a lot about the institutions coming into crypto and that is why we are super excited to share that we are hosting the digital assets Summit we hosted this since 2019 it's coming up in London March 18th to 20th don't miss your chance to get ahead of the curve you can get 20% off with code Empire 20 we'll see you in London uh so welcome everyone I'm really excited to host both Casey and Daniel which I've known for a bit um the idea was to have a podcast that would cover all things Ai and then why it's relevant to crypto of course we've all started to

see companies that have been trying to um leverage some of the key innovation happening in crypto and figured this would be a great time to have both Casey and Daniel who are you know in my mind uh people that I've learned a lot from in the space um and so Casey Daniel welcome I'll let you both introduce yourselves thank you uh yeah happy to give a quick intro here so my name is Casey I was previously an investment partner at Paradigm and before that was an engineer at Google and studied Computer Engineering undergrad and then have my MERS in AI um thanks for having me Santiago Casey always a pleasure I'm Daniel I'm one of the co-founders of a company called modulus we sit at the

intersection between Ai and crypto so this should be really fun uh to be able to dive into this and before modulus uh I I just wrapped up school at Stanford also studying you know CS like everybody else now uh but actually with a specific focus with my co-founder uh in AI so definitely right at the intersection great so so I'd like to spend perhaps the first segment of the Pod really just giving a more 101 102 kind of foundational knowledge of of AI because I feel like most people don't appreciate the key unlocks and Innovation that has happened to to lead us to where we are today where everyone's talking about AI that it's the hottest thing that it's going to be hugely transformational and I certainly

believe it but I I I think that appreciating why and key innovations that have led us here would be a fantastic setup to then um give everyone a good understanding so I'll leave you two really to have that discussion so maybe we can get started with that so I feel that a lot has changed over the last few years and I can just do a quick history from my perspective empirically of where AI has been and where we are today so when I was in school a lot of what we were doing was not ml it was really data science and we were doing a lot of statistical models and this whole concept of an autonomous or like a model that would be able to make decisions was

a little bit sci-fi still and then I would say like around 2017 when I was at Google we started to see the power of what now is called a foundation model and you can think of that as just a big machine learning model that takes a lot of data uh and is trained on that data and so something that I think is really underappreciated is that the internet has basically been this trojan horse as a way to get us Digital Data and our models were historically capped because we didn't have all this information that we could feed as examples and so yeah I like to think of that that Meme of the Trojan Horse and that the horse is the internet and inside the horse is the

data and then on the other side of the wall is machine learning like like modern day machine learning and so kind of like just going back to 2017 we start to see that we start to see oh wow when we pair you know Transformer based Nets with a lot of data with really powerful Hardware we get interesting models and then we kind of kept them at the big Tech Giants so behind the closed walls of Google and meta all of those companies had these these very powerful models but they didn't really open source them at the time and so what's super interesting about now and one of the reasons I'm expanding my own mandate to invest in AI is we really are at this Pinnacle of this revolution where

powerful Foundation models pre-trained neural Nets are accessible to everyone by you know either running them locally or hitting an API and so it's a very interesting we'll look back on this time and say this is a very interesting time in all of history I mean neural Nets have been around since the 60s um but never in this capacity and never to this accuracy and uh it's just it's very interesting to be here today because I truly do believe this is the beginning of a totally different transform transformative age maybe Daniel or you Casey could talk about it so it sounds like it was a combination of more data but also Hardware becoming cheaper that would allow you to run um these models

at scale and in a cost-effective manner and and the algorithms got a lot better like reinforcement got a lot better Transformer based Nets got a lot better the architectures got better but then I would I mean I'm curious Daniel your perspective from my perspective it's like those three things it's better data mainly through the internet better models and algorithms and then also this acceleration of Hardware which we're so so lucky to have now yeah I mean um absolutely agreed I guess maybe of those three all of which are really really important I would say the one that is particularly or has uh particularly been relevant for the past couple years uh at least from my perspective has actually been the the the compute costs or the hardware aspects right because in some sense I mean you mentioned this Casey

but the ideas behind uh machine learning these kind of of autonomous models have been around since the 50s and 60s uh and a lot of these algorithms that we are now putting into production those ideas have also been floating around for 20 30 years um you know I think um Alex net was like 2012 2013 around that period and the question for me anyways is um always kind of what changed between then and now in particular that enable these kinds of massive models uh to suddenly become much more effective at the task that we've always wanted them to do right um and you know I would make an argument around gpus and you know lowering the cost of um you know that silic per unit compute um and all those fund metrics but I think that's a really

important part of the story which has enabled us to bring these algorithmic improvements to our models and then actually digest that data that the internet has provided because I think around the same time 2017 which is when I started college we have started uh at least in the academic uh uh side of things you know moving into computer vision right and and like being able to uh give these models these like multimodal Expressions uh and that's entirely enabled by the fact that Compu was super cheap right like Nvidia had a deal with with uh you know my class so we could just bring up you know more gpus and spin up bigger models and these kinds of like um you know seemingly trivial uh uh improvements to accessibility I would say is a huge part of of what's responsible for like what

AI is today also just on this point since you called out CV I think for listeners that people are less familiar with ML are probably wondering like what is working today and what is not working today and just to concretely answer that question I think that two things that are working are basically CV computer vision and NLP language models and so we don't call them that anymore those are more like uh Antiquated terms we call them Vision models and language models but those are the predecessor Lang like terms and it's a little bit different and there's new and all of this but just to do this in a broad stroke and so just to say that out loud like I I do think that might be helpful these two categories have really

risen the ranks of where we're seeing the biggest gains and the reason is because of everything Daniel said both in hardware and with the gpus and we have the most data on that we have the most data of like text and images so perhaps if you were to talk to a researcher um perhaps five or or even 10 years ago you may not even go that far um and say that he just stopped there like what would he have told you about the progression of AI in that state versus today you know if if someone in Academia just stopped like was it three years ago it just felt even maybe two years ago like how would he have describ he or she would have described AI because I feel like that would have

been quite different I Casey I I guess two or three years ago I don't even think Transformers like like I wasn't taught to me in class we didn't really talk about it very much it wasn't like the the de facto winner of the NLP kind of uh language model discussion at all yeah I think at Google yeah maybe a little different because it was so inread there a little more so but I yeah I think the entire architecture was different I think we were doing a lot of like lstms were still a thing back or yeah long-term short-term memory was a long thing back then but the entire ecosystem was so dependent on different

architectures that are we've made so much progress in consensus of how to move forward I think that's the biggest difference is like three years ago I feel like there were different camps of oh we think this is the right way oh we think this is the right way and I think now we've really created these Stepping Stones where we're all sitting here saying okay this is the state of art today let's all work towards this and then you know we're still seeing of course progression and Innovation but I feel personally like a lot more consensus I don't know if you disagree with that Daniel no no agreed I think um yeah it was very it was much more tribal um back even just a couple years ago right because it was not entirely obvious which techniques would be um would have these like emergent properties that just show up uh when

when you get to certain compute thresholds um obviously Transformers diffusion models like all of these have been um have totally been Paradigm shifting um but uh but yeah it's just very hard to predict when these architectures will emerge suddenly as okay yeah like the this this way of laying out your algorithm seems to make it make the models really efficient at you know logic or really efficient at digesting images um so I think even back then a couple years ago I would have been much more pessimistic about the progress uh in the field but clearly you know it's just gone significantly better yeah I think uh well if I remember U you know there's like Google brain and there's a couple of initiatives

obviously like I like there was I what is it m that like theep blue I believe like it was what is it like the chess algorithm that like Watson yeah yeah and then uh then they applied that to like medical stuff so it's like been percolating um I guess it wouldn't be a podcast about AI we didn't talk about um like chat GPT um and how that has been very accessible for people to start understanding how they could leverage AI in their workflow in their daily day interactions uh what made like U open AI just rise to become this what is probably synonymous when most people I think it we were to survey like a 100 people like what is AI to you they're

like oh yeah like open AI like the GU Sam like chat GPT but of course it's like there's so much more behind that but I'm curious like how like some some background around that like were they pioneering particular thing like what made them what they are today from your perspective h well they are the grandfather of the modernday language model so like let's just start there they are the first company to really offer a language model that's accessible and offers a step functioning utility to the degree that they do and Daniel I'm sure you were playing with uh gpt3 as well you know before it really hit mainstream but it

was good it was okay but fine-tuning was a pain in the but and for people that are less familiar with fine-tuning just a process to basically specialize a model and it was it was just a really big pain as a developer I remember doing that I think in 2021 and just being a little bit disappointed with the output and then when gpg 3.5 came out I mean I think every developer remembers the day that they like tried 3.5 for the first time it's just like oh my gosh this works and the fine tuning like it's just everything was better to a degree of like you know in Venture we think a lot about set functions like

where is the bar where the switching costs become clear and so Santiago to ask or answer your question to me it was like when they basically released I think it was like Da Vinci I forget like exactly the specs of it but there was like a model that they released that was just Superior and superseded everything else in the market and then they really use that to crystallize their Market leadership by basically making it as easy as possible to use offering up a UI an interface that anyone could access without needing any developer knowledge and so from my perspective that's kind of the progression yeah I mean I I think it's a

it's it's this is a weird thing to say now but like AI companies didn't really exist a few years ago um as in like what does it even mean to have an AI company maybe outside of scale AI you know around like the data component it's mostly just products and services um that were enhanced with uh these really hypers specialized models right there there was no such thing as kind of AI as a product so to speak I think it came to or open AI is kind of big quote unquote product level Innovation here is to offer that language model in this really accessible form factor and have the model be performant enough where it's starting to get that almost like general intelligence uh vibe to it right where like my mother uh could hop on um

you know chat. open AI whatever and and actually engage with this product and have like a really um you know productive experience that's just like never existed before right previously it's like oh um I have this model which helps me understand risk better in my portfolio or I have a model which can help me pick out faces in in a crowd um you know they're all part of like a bigger product pipeline um but with chpd that is the product so maybe pulling on that thread a bit more just going back to the kind of the more foundational stuff so the requirements are good high quality data uh and the ability to ingest that in a cost-efficient manner how would you

describe like a language model to your mom to your dad to someone that is totally not technical I would say that a language model predicts the next word or character by taking examples uh and using that to predict so for instance yeah that is literally what it's doing for what it's worth like mechanistically and it's just crazy that it works that well yeah that that that to me was quite eye opening it's like it's predicting the like the next word but not but somehow the con the entirety

of the text like just his very coherent and it's very accurate um to me that was one of the more mindblowing discoveries around how this actually works because I would have thought I mean I think there is some relevance but when I heard that I was like really how is this even possible it's just Li what you pointed out uh Santiago around like how does it kind of keep that context had been like the big roadblock to um I would say the adoption of Transformers for a long time right like oh um this idea around attention this idea around um you know keeping it in memory and navigating context so halfway through a sentence it kind of doesn't just go on a different tangent uh if you look at you know gbt2 gbt even three right this is the kind of

performance issues that you see um and really you know it was kind of the continued investment in that architecture to figure out exactly how do we get these modules correct um or or more performant anyways that Suddenly It's like oh my goodness it's like generating entire conversations with me fully tracking everything yeah yeah and maybe a question in terms of like I'm gonna probably say something that may not be correct but like when it's when the when the algorithm is thinking about okay probabilistically it looks at all the data that it has at its disposal to say in this this particular word the next word to predict the next word it sort of says in the context of

everything that I have I have in the model and the training is probably this word that is most AP or relevant and so then it does that very quickly to compose the answer when you're like doing a prompt for chat GPT right um can we talk about like the quality of these models uh for instance different languages um or the bias in them or when you're trying to apply them for specific vertical and use cases are we at a point where they are as relevant as general prompts or are we in that phase of there's there's an opportunity to get way more sophisticated at like

specializing these models to be more specific for risk or for Bio or whatever it absolutely makes sense it just also covers a lot of ground right so um it just depends on which thread you want to I guess pick at I mean I think specialization is as a good a place as any um yeah I mean you know these models are like I think one of the more surprising things to me anyways is the kind of zero shot um like just like the untuned performance of these models are already very very impressive right I mean X instead of Y right like literally just going to um you know any of the chat GPT prompts that that you might have in mind and just inputting it directly for a response um so you know

uh like you can always tune that conversation by getting more specific uh based on the response you're getting from the model to give it more information but imagine not doing that right just a single query um and it getting really accurate results right you ask a question about something medical or something legal and it finds that piece of information or that relevant context um that is again a very surprising feature uh the the thought for a long time was that these models would have to go through really really intense fine tuning to be able to even get to that level of performance and it looks like actually these like journalist models are already very good even without that fine-tuning but now you layer on the fine tuning and uh you know that performance uh that

specificity goes to a level that again you know three or four years ago I I would have totally been just flabbergasted by um Casey I don't know if that's been your experience yeah yeah I so I agree there's a lot in that question and so I just want to run through a list so the different languages like this is my question I think I like to have you know anybody can access is like when you're thinking wouldn't LM be good at this you just think is there a lot of data of this on the internet and so of course it's really good at every language because there's so much information in every language and so yes that to answer that question then um I feel the question about how good are these models at a

base level and the kind of where are we going next with this specialization I think of 2023 or I think we'll look back on 2023 and say this was the year of Base models just like l1's you'll be like this was the year that we made like ethereum in salana and then it's going to be all right well what are going to be the l2s on this and what are going to be the apps on this and again a crude analogy but bear with me the way we're going to get there is fine tuning and I just again want to redefine this because I am kind of a little bit hesitant of we're both just hyper technical so just think about fine tuning as you put in some extra data and you want it to maybe put guard rails on it that's

one way to think about it to behave a certain way and you want it to uh prioritize certain information and so to Santiago's point of these like bio being a fascinating now specialized use case for ML we're absolutely going to see that in 2024 I mean we're going to see so many different protein models come out in things with like neurotech models biotech models all of these different specializations and we're already seeing this like I think hervey aai I'm not an investor no no incentive to say this but I think that's a really fantastic example of a model that's reaching product Market fit that is using law as its domain and taking a base model and then training on top of it and I think they're actually I don't know too much about them and they're

probably going down the direction of maybe building Their Own Foundation model but the point is that you can take these kind of modular building blocks very akin to crypto and start composing them in a way to build more specialized uh llms yeah on that point uh because I do want to transition the discussion some pointed to how how are these things being productized um and where you see that opportunity and then of course talk about the relevance for crypto um do you agree or disagree around bias in the data problems and challenges that we may face or is your thinking no these models are sophisticated enough that they learn by themselves they improve by themselves to that kind of true

intelligence where the model will just so for inance for law or bio like if you're trying to do proin protein unfolding like there's just stuff that you could have never looked back on because you are trying to cut new ground into discovering A protein that literally we we don't know that exists yeah but maybe it can do a certain relationship um but of course like Pharma visor or whatever Mna might have like data that they're not open sourcing that they have for themselves that they're not going to share with anyone um and so they might keep that all themselves um would you say that like that type of model that has been trained with proprietary data will always be better than a model that is generalized

like um I'm curious uh if if you have a view there uh I think the bias is there because it's in the data so it's yes we have to overcome that and that's why you know chat gbt is not just gbt the base model there's a lot of steps post gbt to take that Corpus of information do rhf on it sorry do like bring humans in the loop to help them uh modulate the output and so yes there is bias to to answer that concretely and then the next question of like will a proprietary data set over always outperform uh a public data set I think always is a really tough word I try to

stay away from absolutes first of all second of all I think that uh yeah we're see a lot of companies that have these data sets come forward and this will come out when we talk about the value or kind of the intersection of crypto AI but there's kind of this elephant in the room right now I feel in that data is this the one of the most important resources in the world and yet nobody is monetizing it in a way that feels uh like robust it's like this thing that you keep and you harvest and our entire internet inter of social internet has been built on this premise of use our products for free but we'll secretly you know mine your data and then we'll advertise to you it's just a

very opaque nebulous kind of Market structure that we work within right now and so I I hope that answerers Santiago but it's like yes there is this kind of belief right now that this private data can be more valuable um and we're seeing that come to fruition time and time again as companies go into Ai and maybe Daniel if you can answer the third part of the question was in terms of Discovery when you're using Antiquated historical data how does that overlay into like how sophisticated is a machine when you talk about true intelligence to making breakthroughs like protein like discovering new proteins um or you know for law well you're looking at all the precedent that

exists out there but of course new precedent gets set every day or you know every so many years in the Supreme Court I'm curious like in terms of the the Discovery piece MH how much of AI today is just reminding ourselves of what historically has been done with that context of data versus totally new stuff totally new relationships that we have never conceived as humans before yeah I think uh one of the first things that Ryan and I did uh Ryan being my co-founder when GPD 4 became ex accessible was we wanted we asked it to solve one of the Millennium Problems or you know try to give us the proof to

canor is like you know one one of the um the the proof-based questions that we would get in our math classes right uh just to see if it could string together enough logic to either fill in things that we've already solved and therefore should be somewhere in its data Bank although maybe not frequently uh or something that we've never solved before right just try to figure out is predicting the next token the next most likely word in a sentence enough um mechanistic sophistication that it can actually solve these problems s that we thought that we could never solve uh and and the short answer is I gives it a really good try um no unfortunately we didn't solve one of the one of the Millennium Problems that day if we did that that would make headlines um and I

I guess so so there's two I'm of two two opinions here one is um pattern recognition to the level or the the sophistication of these language models has never been accessible or or has never been this accessible before right so certainly I would say there's a whole class of problems uh you know maybe they're they're not quite Discovery or they're borderline on what you classify as a discovery that will be unlocked because more people can you know go at those problems with these new tools uh but there are still I think problems out there that are absolutely novel right that requires a kind of Ingenuity that we've never exh exhibited before or have done so very infrequently and so those kinds of architectural features is very

hard for the model to inher parent or to pull out uh that kind of like semantic information um you know is um it just requires I think a really high degree again of sophistication with all of that said I don't know what gp5 looks like you know Gemini looks great I'm sure you know the next version is already being developed it's quite possible that uh although so mindboggling uh to me anyways that a a deep understanding of language itself turns out to be this incredible tool to just solve a ton of problems uh I think that would be like the strange unexpected discovery of like the century right um so uh yeah not a very

satisfying answer there's some tension here uh even for me um but uh uh but I I do think that I wouldn't be surprised at all if tomorrow it turns out I'm totally wrong and we're solving new math problems or or you know unlocking uh you know protein folding or whatever it is with just more sophiscated models yeah can I say one more thing on this of course yeah is that I feel something that you're hitting on is basically asking is like look how smart are these models actually and I do think another way to dapose the past to the present is in the past we used ml to predict things so like probably the most well used or most used application

of ml is recommendation algorithms I think because we use them every single day and basically what you're doing in a recommendation algorithm is predicting what you'll buy or what you'll see or what you'll stay on and that's yeah basically one form or one expression of ml now what's interesting about 2023 is it's no longer yes it's still predicting in the technology part of it but the actual use of it is not prediction it's creation and that dros is really really interesting because you're not using it as a developer to protect what somebody's going to click on you're using it as an end user to create code to create images and to create writing and I think th for for me like I think

those are three of the most popular use cases right now and so to Santiago's point of like can it really you know can it really do things that you know we can't do or we can't predict and I agree with Daniel it's it's a complex answer but I just want to call that out because that to me is one of the most inspiring things about where we are today is it's not it's kind of broken free of that and it is in kind of this like creativity Innovation space uh and that's just really remarkable actually if you just think about it from first principles yeah I think that's a great segue into the productization of AI then eventually going into crypto but uh you know as an engineer as a artist uh you know

architect whatever um where are you seeing the most amount of usage um and like if efficiency gains that you talk about Casey is like hey if you're a developer you all of a sudden can use AI as a companion kind of tool to just superow you know your skill set and human plus AI is just just remarkable uh or just purely AI is better than human but I'm curious like where are you seeing these use cases pop up and where what are you mostly interested in sure I mean I can start very granular which is you know if you walk into the the modulus office today um in a different way work uh you would see that everybody has on their monitor pulled up um some version of a language model either

co-pilot chat GPT whatever right like we no longer develop uh alone which is kind of odd right like oh there's this bug that shows up even with these like in our case you know cryptography which is very math heavy very logic heavy you know these areas that you wouldn't expect language models to be a huge resource in we still use them because they're this incredible accessible um you know first instin to check against ah do you have any ideas I'm seeing this error or how would you solve this problem and so it's lowering the bar just across the board uh for all of us uh on the team to being more productive and being more creative and and kind of solving problems faster so that's one very specific but just like immediately like happening uh uh uh you know example

of where LMS are already enhancing our workflow yeah I I I don't know if this is common knowledge but I think software velopment is one of the most disrupted industries by Ai and I only say this because I was an engineer at Google and I think I can it's like I I think I have the ability to but we're not we're not quite there yet but we're almost at a point where AI can replace like an L2 Google engineer and so I mean that's insane when you think about the repercussions of that and just think about like how that will change Innovation speed of innovation uh and and even more over when I'm dilen in

Founders uh I will basically not I don't like hard and fast roles but I will take a very hard look if they're not using AI to augment their software development process because the way I like to think about it is like if anyone's played Mario Kart you know how when you get that star and you're like a super you yeah that's what it is like right now if you a developer with AI using Ai and a lot of people it's interesting you're not a developer you're like oh gosh the developers they're really at risk now they're just be automated away that's actually not true right now I think that's one of the maybe that's one of my contrarian views is that software Engineers are more powerful today than they've ever been and it's going to be like that for a while where like even though now we have

that you know level two engineer being replaced we're actually just so much better at our jobs um and so we can build what used to take months and weeks it's hard double down what Daniel said instead of instead of needing 10 Engineers as a startup like modulus you can you can have one or two very good Engineers leveraging co-pilot and just becoming like just hugely efficient yes with that said everyone in modulus don't worry you're fine we need you you know um but definitely I guess you know the economic argument had always been look there is this massive demand for developers just across the board which our education system our uh work uh training programs are not able to meet

uh that that demand not even close right but here comes a tool which KX as every specific developer right whatever their role is within the org and I think it will broaden to just beyond developers who who even you know Executives or or folks in you know marketing and all that um it'll look a little different but concretely today it's already happening for you know software Engineers as an example we can now finally after 30 minutes I know most people come to Empire to discuss crypto so um you know maybe we can start talking about crypto and how a is relevant um and and you know of course Daniel and full disclosure both Casey and I are investors in Daniel's company modulus so um you here is really excited about what

he's building well there's obviously bias but anyways none of this is any advice um so Daniel maybe you can kick us off in terms of give us the story behind your thinking around okay I want to do something in crypto and I want to apply AI to it yeah for sure I mean this is going to sound increasingly uh you know ridiculous it may be the most dated thing that will be said in this podcast and that is that uh about a year ago when we first started modulus there was Zero AI activity in the sphere of crypto right and I think that's just going to get more and more ridiculous again um but it's true right and part of that is because um uh and I think it's really important to acknowledge this there is a

lot of Personality tension between these two um you know Technologies right just like architecturally what they mean for the world how they like to be they're quite different right uh and and the other part is quite frankly you know the crypto industry is is a younger one and never mind the fact that AI is just now really coming on the scene in a big way but there wasn't a lot of AI competency when it came to crypto uh as you look across the sector right so when we showed up uh and as a reminder at that point all we knew were the kind of AI training we had received uh at school as well as all the crypto kind of bite papers and papers we were reading in our free time we thought hey you know we see this interesting technical inflection in the context of zero knowledge proofs

don't have to get into that right now but tldr this seems like the perfect Bridge or the perfect tool to bring this uh you know emerging discipliner on AI to augment and enhance the crypto world you know we were always hearing about like ux issues and and kind of like you know it's a high friction and it seemed like AI features everything it represented from recommendations all the way up to llms you know was the perfect antidote so that was the origin story basically the Inception of modulus very simple just how do we bring these two technologies together uh so so that we could enhance crypto but still honor those two personalities and what they're really really good at right and our solution that is ZK uh but there are

others and you know happy to dive in to any of that where it's relevant well I guess maybe Casey from from your perspective because I always tell people that you've been um as far as I can remember you were thinking and researching and studying AI before it was cool before it hit the hype cycle and McKenzie once Mackenzie starts writing about it it's sort of game over in my mind um and so um like from your standpoint you've been a crypto investor for for years um and in your position like how do you think about that and how how did modulus become in like why did it become interesting to you yeah and others but we can start there of course

so 30,000 foot view veryy think to say e he here we go no but seriously so AI is one of the most transformative Technologies we've ever made ever and I've known that for a long time since I've been a kid and it's an inherently centralizing technology and the reason it is if it's not clear is because you are more efficient when you have gpus connected collocated you can do more from a fundamental Hardware level and so this has this huge ripple effect of basically

centralizing the entire thing and as a believer of decentralized Technology I've always felt that one of the biggest threats we have at honestly not to sound super philosophical but Humanity faces right now is centralizing AI to a point that the Innovation doesn't compound and so that was the scariest thing when I was at Google is like holy cow we have all of this to ourselves kind of situation and the reason it's so important to bring these two spheres together to Daniel's point is you have decentralization which is a naturally decentralizing Tech you have ai and that might be the best mechanism

or the best method to keep keeping AI open and accessible that is one of my biggest goals of what I'm trying to accomplish and so when Daniel and I originally connection I mean modulus has come so far so fast but I remember you guys were one of the first ones to put like a small neural neton chain I think and I had played with it and you know whatever you came up from Stanford to visit and just the interest or kind of this Meeting of Minds thinking okay the they're these Technologies need to play nicely and yes I know AI people maybe don't love crypto people but we we need to start bridging these worlds because it's going to be one of the most

important things to do in our lifetime and so like I can dive into kind of like how I view the market but just that's like the substrate to how we've gotten to where we are today or how how I feel yeah let's do that I mean I think the important thing that I heard there was inherently technology is centralizing force crypto is is sort of pulling that back and trying to resist that natural gravitational tendency for things to centralize um you can be idealistic which I think everyone in crypto to some extent fails because you then don't become pragmatic around yeah we can talk all day about how nice it would be for technology not to Cluster not to um you know become centralized but the reality is economically or not

that this just may not even we can hope for that but can it actually work so um maybe this could be a good time to to unpack a lot of what you said case and Daniel around you know specifically like what is it about zero knowledge proofs that make it relevant make it actually powerful enough to apply that uh in others yeah yeah I think Daniel let's start with just like onchain verifiability and what that unlocks and maybe yeah if you want to that's like one very very exciting use case of this intersection proof of humanity or whatever you know I don't with your keys I don't know sure sure I mean I think just stepping back for a moment one way to think about ethereum or or you know take take your blockchain network of

choice but ethereum for the sake of this argument uh one way to view at look at what it is is this incredibly incredibly robust public um record table right where you can just log whether it's transactions or or or you know state or whatever uh where everybody can check and and take a look and go okay yeah yeah I can I can come to a relatively high confidence belief that what's on this record is accurate right um and so this this this the capacity for this network to be this like great uh uh you know Common Ground to the world's uh kind of appetite for high integrity things uh is a big part of what you know we try to celebrate at modulus right um

and I guess I don't know if now it's too early to kind of take a peak Beneath The Hood At what Zur cryptography is and why we think it's the right tool for this um but just to give a super super succinct I guess um framework for thinking about things um zero knowledge proofs for us is a capacity to make account of things and the blockchain network ethereum is just where we log the accounting basically right and all of this is happening in a really trust minimized way and we think it's an excellent tool to do exactly what Casey mentioned earlier which is to counteract some of the centralizing influences and tendencies the nature of something like artificial intelligence um but case maybe you have a different take on

public verifiability that uh you know we can also work into you know the framework well I think something maybe just recent that happened which is pretty interesting is I don't know who's read Biden's AI executive order but it's worth reading if if you haven't I also have a summary if that's helpful but one of the most potent or Salient things that I took away from that was the government sees the need for Providence moving forward and I verifiability is one of the methods that may achieve that and I think there's a huge set of challenges with it but I think I it's it's one of the things where I think if you're playing the pro

the pro case of this is like it's not a problem looking for a solution it has already by the government and by like the general public been identified as one of the biggest problems with like deep fakes just like model accuracy uh model credibility and so having a system that's accepted uh to verify them makes total sense moving forward and I think it's really really dangerous to predict what's going to happen in AI next this is like one of my more interesting or something that I've been chewing on recently is just that AI has basically been the reverse of what we expected so like in the beginning of AI we really thought that or at least like my cohort thought like reinforcement learning of robots was going to be one of the big

first killer acts we had this kind of Revol Revolution with like Foster Dynamics and in that whole era and what ended up happening was robots still are extremely hard to control and what we one of one of the first killer use cases of AI is human empathy and relatability and uh accountability and those are the things that we thought was going to happen last and so we're having this like total paradoxical thing with AI where the things we thought would be easy or hard the things that we thought were hard would be easy and so I say this as like a caveat of let's have the conversation and I'm happy to tell you what I think is is interesting I also think it's really dangerous to be prescriptive with such a new technology new in the capabilities

um yeah and and so the the verifiability the Providence of that I mean of course the simple let you know lizard brain of Mind thinks okay blockchains are data Rich to your point Daniel the applicability of blockchain is such that anyone can verify this is what makes blockchains block if anyone can go in and verify the state and the transition you know per block um you know then then that makes it really powerful because it keep keeps the system in check it's open it's accessible it's transparent not only that but over time if you think about blockchains they will produce an inordinate amount of data that it could be used for training um and and other things and so and the data is sort of

pristine like you have these guarantees that it hasn't been tampered with you you know yes of course now most of it has been transactions you know you know sending Bitcoin from one address to the other but that it doesn't stop there um I think you can embed a lot of data into these blocks that can make it useful so um back to maybe zero knowledge proofs or how you're actually thinking about um you know modulus as a product like what what is the product like what what exactly are are you selling um or or building I guess yeah I mean you know one thing that uh I think is packaged within U the nature of what machine learning models are in production you

know we call it these like centralizing influences is also the fact that they're like blackbox models for the most part they're very opaque it's very hard to predict um you know a specific set of behaviors it's very hard to point out fraud when this happens or any type of manipulation whether you know intentional or not never mind bias and all the rest right these are all like active challenges when it comes to machine learning and so when you think about applying AI models uh in in production in the real world those risks just comes bundled it now it might not matter so much if it's you know the model is trying to pick out a nice outfit for the day or you know having a conversation with you about the weather or anything like that but our belief anyways is that it will really matter

when the stakes are high be a financial legal you know medical some of the contexts that we mentioned before uh and one uh you know very clear Beach head for these kind of high integrity environments is blockchain based applications right so if you're running a DEX you're trying to you know uh you know balance some pools anything of that nature um here is exactly where that attack Vector will be really present so to answer your question Santiago what is a product we're working on well we're using zeron knowledge cryptography to basically prove that an AI model executed the way that it was supposed to there was no manipulation there was no tampering even if it's a blackbox model so to speak it sits in that box and nobody kind of proded it to act in a

very bizarre manner or in a tampered way and we make that result something that is both verified on the blockchain so you know from now until forever anybody can verify that that specific AI result acted correctly right in accordance to the pre-agreement um and secondly making that result uh you know native to the blockchain basically uh something that can be digested by any dap or composed with you know with all those properties that we really appreciate about operating on chain so we call this idea zero knowledge machine learning or zkl uh and you know it's early Innings but we think it's one of the most it will be one of the most powerful kind of um you know building blocks to not just blockchain services in the future but

how AI is used just Society wide I think that was so well said and I think one one thing to just put one step further is once you have this which gets me really excited is that it unlocks the rest of the crypto ecosystem so if you can verify then there's all these services like D5 for example that you can plug into you can't really do that if you can't verify it I mean you could but it's like it's a a total unlock and so it's valuable for the output and then it's also valuable for what is on the other side of that verifiability exactly I mean you know the the naive way of of of U naive mathematically not to make light of anybody's you know execution

but the simple way quote unquote to do this is to just write your AI models on the blockchain um but if you're familiar with gas fees and congestion and block space all right everyone so we talk a lot about the institutions coming into crypto on Empire Santi and I are both headed out to London March 18th to 20th for blockworks his 8th ever digital asset Summit Das this is an Institutional buttoned up conference that we've hosted since 2019 I like to joke that it is probably the last remaining in kind of suit and tie event in crypto people are still wearing suit and tie it's pretty funny but you'll actually hear from a lot of the largest institutions in the world coming from Standard Charter FIS JP Morgan framework

folks coming out winner mute vanc golden Sachs there are a couple big themes of this conference one Bitcoin Catalyst the having and the spot ETF two A View From the buy side three rw's tokenization and stable coins four Global regulatory Frameworks five institutional infrastructure including Bank and payments and six the macro case for crypto if you have anything to do with the institutional side of crypto you have to be there Santi and I got you back we hooked you up with a 20% off code it is Empire 20 there is a little competition running internally at block works to see who can drive the most number of tickets so help Santi and I out register with our code and you get 20% off that is Empire

20 I was going to ask you just that you know we can as far as I can remember zero knowledge Pro have been this sexy holy gra but we always bump into how is it cost efficient can we do this at scale exactly yeah and actually Casey mentioned this when we first met we had just put a tiny tiny tiny new network on chain using zeron knowledge proofs so the model is running on a server like anywhere else um but we prove that the model execution was correct and we put the proof in the result on chain but even you know with this like scaling technology we were limited uh to to small models right we had spent a lot of this conversation talking about llms and we think there's a lot of applications to llms to especially high integrity

results of llms to the blockchain network but the big question is how do we unlock you know access to these massive models these highly performant super expressive models with this zero knowledge kind of uh you know framework or zero knowledge path um so that's what we work on full-time basically just trying to figure out what is the right way to make that unlock happen uh and open up access to these really exciting developments in the AI world for the onchain audience especially Casey I mean a lot of what you and I talk about um you know we've known each other for for years and and invested in crypto together and I think you have a uncanny eye for as well you know because in crypto no I'm saying this

with with you know crypto is one of those spaces that is is so generalized and so exciting maybe like AI where it is easy to attract certain type of builders that you know there's a lot of fluff and there's a lot of out there and look anyone listening to this might have come with a certain degree of skepticism around here we go crypto is that industry that anything that is hot crypto people are going to take and just reformat repurpose and try to extract money but not actually build real products yeah that's just the critic if you were to ask most people that are outside of crypto that's what they would say how real is this like are we going to see this in production uh is the

opportunity there when I say opportunity it's not the ability to make money because we have cardan of the world that have not shipped and have you know valued in stuff that yeah anyways how real is this and how um like when you think about the opportunity versus investing in other areas of crypto that are perhaps also exciting how do you see this as an investor yeah so my answer would change depending on what angle I take if it's like the AI first or the crypto first and I'll say for every one Daniel I meet there are 99 people who were just trying to exactly do what you said which is take

crypto and attach AI to it and capitalize on it and so like I think you and I have seen collectively like hundreds of decks yeah so I see one more decentralized GPU deck oh my God let me just say that Daniel is like a very like needle in a Haack situation and we we live to work with people like Daniel right who are really here for the right reasons and want to see the change in the world that we want to see uh how real is it let me just this is like my dependency chart in my mind AI must stay open it must we have to we have to do that as a society crypto in blockchain decentralization web 3 all of those are

very good contenders to help us achieve that and so how real is it today there isn't too too much I mean and I'll walk you through let me walk you through like the six things I'm looking at right now but we're very very much at the beginning of what this intersection could look like and you know you're going to be too early too early too early and then you're going to be too late and I think that transition is going to happen really quickly and that's why I'm spending time here is because I it it is one of the most interesting asymmetric I think opportunities we have in crypto and I think it's a really good contending technology for AI and the principles of web 3 are going to be leveraged uh it's

it's just Cardinal to make sure this technology reaches its full potential does that answer it in a in a way that's satisfying Santiago well it could be satisfying to me but is it satisfying to the world yes I I think what you just said is very I believe it the two things that you said that stuck for me were crypto is a very probably the best Contender that we have to keep this technology open and to resist that tendency to go and and centralize of course you could do that the other way which is by enforcement through some sort of act uh but that hinders Innovation locally but not globally that could you know you could stop gpus from being sent to China but

still the technology always finds a way um and the second one was I think um you know crypto uh is open source and continuous like the innovations that we've seen in crypto like an AI have felt really slow for 10 years and then very fast and that is just inherently I think the state of technology and I agree with you I'm spending time on it even though I'm not as sophisticated and so I lean on people like you to help me understand this space and weed through a lot of the crap that's out there well I I can personally attest to I think both of you being extremely high quality operators by my my impression so and with that uh you know set aside I I do

want to make an argument here as well though that I think the Nate here results in like a product with a capital P that is just also better like period end of sentence right um and if you don't mind me spending like a minute or two on this like mental framework um you know we spent a lot of time in the world of cryptography and it's worth thinking about what is it that cryptography is doing for us in the context of zkl but also just like Zer knowledge in general right and uh actually I think it makes sense to visit just like what AI models are even right like like for me anyways like at the extreme what an AI model ultimately is meant to become is a capacity to scale human decision- making

just like forever right like for free basically right like wherever you want a human to be making a DEC decision you can bring in a model and have that happen for you but with you know even higher performance which is weird to say about a human being but nonetheless right this is very exciting of course but it's also a little bit problematic like this idea right because let's say that you know Casey uh Santiago we walk into a court of law and there's a judge giving us a sentence right the three of us committed some terrible terrible crime right except we didn't and we want to make sure that the judge is making a fair decision right the way we would do that today is we will look at the record of this judge we look at all the decisions that they have

made before we look at their credentials where did they go to school right what law school uh we look at their their habits right any scandals whatever right because human beings have this incredible quality uh of of being really discreet like here is a person right and there's like this web of infrastructure and trust that we can use to evaluate a person based on their past Behavior compute suffers no such luxury right compute just Flats out in The Ether right it's model V1 it's model v1.1 right we change a couple parameters we tune some you know some inputs and the output can be radically different right um all that to say that for me anyways one of the big opportunities here where the convergence of crypto and AI

literally makes the output better is where we have models that are now accountable right that we can look back on the history of that model we can test it for robustness for bias we can look at all the you know llms which is Consulting this lawyer uh all all its past recommendations and make sure that it is fair and it's not make hallucinating case studies and whatever right um and so I think you know taking away all the kind of aspirational ideals we have around decentralization taking away uh kind of where we're concerned with uh uh you know the progress of technology and how it's going to inter interact with Society just looking at this product now right uh an AI model which is now much more accountable thanks to the work that we're doing in

the crypto World um I think we it's it's you know for me anyways it's like okay this is a future that I actually think is better um and so that's a big part of why I think you know um this this stuff is worth engaging with and work worth working towards yeah on the verifiability piece I totally agree with you like if you can have a high degree of certainty that the model did exactly what you wanted to um and you can trace Providence of of what resources it consumed to get to that output then it becomes highly relevant to particularly for the example that you U described around you know you're sentencing someone there's inherent bias you know there's maybe an AI model can help us compare the decision of a human to the

one of an AI that may be less biased and then you know contest that outcome um I'm again going back to the the the the challenge of crypto has always been scale and doing these things effectively you my understanding is you've been focusing and tinkering on the ethereum blockchain now there are competing um l1s that um you know are more performant or claim to be more performant more you know I'm I'm curious in that decision we don't have to spend a ton of time on it but I I'm curious if you have a view on where we may see these applications actually work at scale and be an actual product with like profitable unit economics if you will yeah that's that's a really good question I mean I think

you know concretely um we spend basically all of our time reducing costs right increasing performance right uh we want the the ZK death to be as little as possible so that we can bring more performing models to these Services I will say almost all the partners we work with today uh are on ethereum uh and the reason is actually even less about the performance of the underlying chain and more about the culture right the culture of ethereum today although I'm sure this is true as well on many of the other networks is that of security matters right Integrity matters right we want to make sure that you know we have a stack of smart contracts that compose the foundation of our service and actually we don't breach that security that's provided by that by suddenly referencing

an AI model that is centralized or close Source right we actually want to bring the ZK in so we maintain that security this is something that really really really matters the ethereum clientele or the ethereum audience um and so we found a lot of cultural alignment there that is not to say that that is the extent of our work right one of our other partners um is worldcoin in the context of that collaboration oration uh the proofs aren't even settling onchain we're actually just trying to convince the world coin servers that when people attest to their IRS scans or attest to their identity uh they're not doing that with a manipulated Iris or a manipulated identity right so um you know early Innings we're really just at the beginning but my expectation is if we can do a good job here right by showing

that high security high integrity AI matters and improves daps across the board we'll see that pop up in ethereum and salana and all the other eco systems um Casey would love to get your map of how you think about all the different you've seen many decks some of which are you know as you said decentralized kind of compute networks and uh something like how are you thinking about the evolution and the opportunity set um in crypto what's your time Horizon who are the customers here are they Crypton native companies like worldcoin or may this be a beach hit into actually pulling in uh less crypto

native people um and and maybe them not even thinking or you know thinking that it's crypto but just being very focused on oh verifiability is something that I can really wrap my head around and I like don't tell me if it's crypto just sell me the product yeah so I do think that was spoton and that one of the more exciting things to your question earlier why spend time on this category this might be the most successful category to bring non- crypto people into crypto and the reason is because if you look at what are the killer apps of crypto today in my opinion like the top two are beyond all ones are nfts in Defi and both of them are inherently Crypton native you know

they weren't from The Real World they were built on chain and this could be the first sub sector to you know be prominent inable run where it's not crypto native it's actually coordinating things that are outside of our native world and that is so exciting but yeah let me tell you so I keep a list of the categories of Dex I and kind of the the sectors that people are going after and here's what I'm seeing the most of and I'll I'll try to keep this brief and I can provide commentary on kind of the pro and con case of each of these but the first thing as I alluded to was decentralized networks of gpus the case here is could we build basically a crowdsourced uber Airbnb for

gpus and from a technical standpoint it's it's hard because of what we said where you get so many compounding Effects by gpus being collocated but there's definitely a case that certain use cases that can handle certain latency and just take more time to train that could work um so that that's actually quite interesting and then then the next thing I see a lot of which I love is local and peer-to-peer inference so if people aren't familiar with inference inference is basically when you get a result from the model so you do the pre-training and then you run inference which you can either hit hit an end point or which just anything that gives you back the result what we're seeing with some centralized companies

like olama which allows you to run llama 2 mral other open- source llms locally that's becoming a huge trend is that people say you know what I don't want to send my data through this hugging face or replicate endpoint or you know I don't want to have to rely on Wi-Fi to hit this model there's so many reasons that they're like you know what I just want to run this locally I just want to run this on device and that trend of edge Computing is not only just happening but it makes you think well what's the next step of edge Computing well it might be peer-to-peer inference and that's something that a decentralized network would be phenomenal at and so that that's super super cool uh and then the next category

I see a lot of is just crypto incentivized Behavior so think about crypto incentivized AI apps where basically we use crypto to incentivize our bootstrap a network crypto incentivize rhf mlops all of that kind of category which I put I can do SUB sectors but it's basically just taking what you know we're one thing we're really good at is promoting certain behaviors and taking that property and mapping it to AI then we already talked about verifiability and then the two final things I see a lot of that I'm excited about are uh like uh I would call it self-owned models and data and kind of to the point of data being this really lucrative yet unregulated asset how do we use crypto and blockchain to

be able to give power back to the people who actually create the data and it's not it's not about power it's about it's about so much more than that but basically yeah no no you go no it's more like agency like understanding who's using your data and oping in or oping out and and actually getting some monetization out of it exactly and you can imagine a world where just like metamask today we take our metamask from crypto app to crypto app to crypto app you could do the same with your data and models you could use it on Facebook you could use it on you know whatever the next like stability AI is whatever it is and like it would be this portable thing that you take with you and so that's that that to me kind of makes sense if you squint there's also a lot of

problems there and the last category I see a lot of is like agentic use cases where basically for people unfamiliar with agents they're kind of like autonomous ml models that can make decisions uh and they in a lot of forms are always on um they operate in the background in a lot of ways and so you can think of how it might make sense for AI agents to use crypto natively as their payment um because they can't really kyc and that might be part of the agent infrastru structure that becomes fortified in 2024 yeah the the thing that that I for me was very exciting was this idea of proof of

humanity um you know you control your keys and you can attest to um you know hey I'm human in some way shape or form and I think you can do that um with you know cryptographic keys if you will like somehow um I guess the going back to all these examples while I I agree with them but what would um you know we can like what would derail the case for these um like what are the cons um are they just purely performance costs like if you're a corporation you might wish and hope that you have more agency or it's more decentralized but at the end of the day like if if a centralized provider is just more cost efficient and

performant I think humans time and time again make these trade-offs everyone uses social media whether they understand that their data is going to be monetized you know it's a free product so ultimately that wins For Better or For Worse um you know this is an option but in terms of Market size do you do you see this actually competing against the centralized solution yeah but we need we need more information we need to see right now a lot of the crypto providers that I talked to in the for the decentralized compute for the use case you're bringing up they're subsidizing costs and that's not sustainable you need to economically be the centralized providers or you need to provide another value function which enables the switch and so I completely agree with what

you're saying let's talk about decentralized Compu networks because it is it is a Hot Topic right you have IO and you have AOS and you have like all I've seen like 10 decks in the last two weeks the the the the the hot tldr is it is extremely hard to get access to like uh gpus it is it is a hot commodity right now you cannot find it and so if you're a startup um you just raised a bunch bunch of venture money you want to give it to Nvidia but it's going to take time uh you can have this as Airbnb unlock this idle inventory or file coin right like distributed file storage systems is let's rent it out and use all this unutilized and crypto is really good at

coordination so you can all of a sudden if even if it's not cooc you can you can get the similar performance and there's a whole subset of people in the market right now that don't even have access to gpus and are going to now have access to gpus and they're want to pay for that and so you're just creating a new market opportunity how real is it like how um viable is that is it just a short-term bottleneck you're fixing or is this actually an enduring product over the next couple years I I think it's less real than people think I'm gonna be bold I think that there's a lot of centralized providers out there like vast AI that I can hop on and get a GPU and yeah maybe h100s are rented out but you may not need the the h100 is

this Nvidia like Ferrari right yeah no one can get access toy right but it's the the if you want the Ferrari for this stuff it's the h100 right yeah yeah a100 are pretty but yeah so those are hard to find right now but there's a lot of like GPU marketplaces there's a lot of centralized providers too that are doing this and so you can get on like banana Dev or like runpod or I mean I could blist a million of them and yes then you have theashes and the io. Nets and the renders and all of them and uh like bit tensor I think in the future uh but the the thing that needs to be proven is that you would rather go with a decentralized provider than a

centralized Marketplace and there are some early signals of that there are actually some interesting CA uh case studies of technology companies that I won't dox on the podcast but they're actively choosing to go with with decentralized providers for a variety of reasons but we need to see that sustained and crystallized because that that's the most interesting thing I've heard over the last month is getting on the phone with these tech companies that are you know based in the valley they're just regular AI companies and they're saying yeah I'm going to go with this decentralized GPU provider and I'm gonna end my contract with you know my other burst provider and here are the reasons why I mean that's like that's isn't the the biggest reason I've heard consistently across these companies is like it is very localized so if you want to

have access to a GPU that's in Europe you have to have a different service contract and just the time component becomes faster from a decentralized standpoint because you sort of transcend local yeah I mean you don't have YC you don't have to do anything like it takes me two seconds to spin up a GPU on a cost so there was like zero friction in that and then the other thing is just for people that are scared about the future of AI regulation this is like more of an ethos thing but if SE if CEO is scared about where the us is going with AI they are like much more apt to go and invest in decentralized architecture because they just want to be safe Dan you've been quiet in the background but um I I I mean I know you

may not but I want to get your perspective like as a builder where you see beyond what you're doing other opportunities uh maybe other Founders you've interacted with of real maybe the real more question is if you had to Pivot today into something else crypto AI what would you be doing yeah I I maybe just because we're on the topic of decentralized um you know compute networks and so forth I would say uh the challenge actually is not so dissimilar from ours right in the sense that there's like a science challenge here right which is that I think Casey is totally right here and that fundamentally um I guess I I this is my sense of things in general anyways

ultimately it it's just about like a better product anyways right like um if I'm a um you know if I'm if I'm running a startup and I need a GPU cluster to train my models on I'm going to choose a provider which is cheaper more robust you know maybe not more not as geographically constrained and so forth right and if that happens to be a decentralized network I don't really care what's on the back of the box uh you know the front of the box is good enough for me right let me sign this contract and just get rid of this problem and so in so far as uh you know there's another player and space for example called Jensen right which is trying to do this like decentralized GPU training uh Network they're more focused on training right not information yes they're focused on training um and and

and specifically of training you know large models um and in that case like although the the cryptography is complex it's a very linear challenge right if they can get the cryptography to a low enough cost uh standpoint or low enough unit economics so low that it's better than the centralized provider you know I just raised a billion to build some cool llm or Foundation model why wouldn't I choose this decentralized network right it's it's 20 to 30% cheaper than going to Amazon or Google or whatever right all that to say that um I think that the funny thing is ultimately the best product wins right over a long enough time Horizon and that's what we are about right it just just it's just that the cryptography or or or the crypto aspects of a lot of these decentralized

networks are so early they're so they're so in their infancy right that we need to oftentimes juice these incentive systems uh to try and get people to migrate to our networks but uh to Casey's earlier Point that's not sustainable and an ofus skates uh you know product Market Fit U which is very very dangerous like you get addicted to the to the drip right um so to answer your question Santiago if I had to Pivot and work on something else today um I don't know what I would be doing I'm I'm absolutely obsessed with our problem statement uh but it would also most likely be at the intersection of AI and crypto just because it's such fertile ground for so much and I think being focused on the science uh those kinds of unlocks are meaningful and

moves the whole industry forward yeah I want to emphasize something you said which is incentives and it's something that is obviously more people are critical about these days but I don't think there's anything wrong if you use incentives the right way um as a pool mechanism and once people because some a lot of times these networks become more efficient um and so but you need to Kickstart them and if and and you solve that like that like friction to get people in the network um and then once that's the case like any any platform you need to solve for supply and demand so like if you can do that with incentives like uber was giving away free rides like you know and then subsidize like this is the way networks get off the ground to like resist the inertia of like switching and stuff so

there's nothing wrong with crypto incentives I feel like somehow that has become taboo and I think people really missed the point like that is the most powerful thing that we have is coordination in a more efficient manner um Casey I want to perhaps go back to something you said which is um a lot of times you know as particular you said it about AI is things will look like they're slow and then all of a sudden you believe that there was going to be a very fast unlock and development around these things um as as uh and I wanted to ask both of you this question is like as you think about the next two three years in crypto or just in general like where might we see those huge unlocks uh whether it's specific to modulus or

other things how does that impact crypto overall um and specifically like um crypto AI you know Market yeah I think the biggest thing we're seeing in AI is or what we'll see is the productionize of it so it's cool and everyone's having these aha moments this year of chachu BT and you know this first agent but they're not really production level ready and that's maybe an under appreciated fact and so this I think we're going to see a lot more robustness around us bringing these into Enterprises and so this is like a total in my opinion a retail driven phenomenon

right now like AI of course there's a lot of Enterprise applications but in terms of like what people are talking about and excited about that's retail and so what needs to happen and like not to get t here but what's so different about or what's so different about AI is that the whole thing is non-deterministic meaning that you can't say what the output is going to be it's it's a basian situation and when you justess was that with traditional programming those are rules these are logic these are pure functions you know what's going to come out and so from a programming standpoint the whole thing in my opinion kind of needs to change and this will loop back in that the reason things are toys is because we

don't know fully how to handle the non-deterministic nature we're figuring this out on the fly like the amount of things I have break on a daily basis because I think I have this agent running for me and it just runs into this Edge case and then I don't know how to handle it and so once we figure out how to put the guard rails up in a way that we can really trust these systems and say look if it does hallucinate it knows what to do and you know you you can really really trust this that is going to unlock the next level of use cases I think and and like so that's just the answer your question and then how crypto will play into it I mean again I think it's dangerous to predict here but it's just like I think that needs to happen

before we can even have that conversation because to me that's like the necessary next thing for us to advance Daniel I'm I'm so curious what you think about this because yeah I I haven't said this out loud too many times no no it's yeah it's definitely one of those like underappreciated features of just AI in production which is a non-determinism um it's a challenge when it comes to ZK approving as well right uh just explaining the right like smart contracts the way they're composed is like like they're they're super rigid that's the whole point right like super predictable you know exactly what's going on um and you know in our world uh we're we're entirely obsessed right now with how do we enhance STS with AI right and so trying to bring that degree of

uncertainty into a regime right some some like these two should never mix because of this right but try to do this in a way that's responsible high integrity and again you know uh respects the ethos and the personality of both disciplines is really really tricky um but I guess to answer Santiago's question in terms of what what I'm looking forward to in the short term um it's it's exactly that that convergence right I think uh 2024 we're going to see a ton more daps with AI features natively just like part of that service stack right whether it's recommendation whether it's risk modeling you know we just started working with a company called upshot on you know appraising nft values yeah yeah and automating these kinds of markets or a project called Ion

on digesting like the risk profile trying to predict slashing with AI models uh and then you know issuing um table coin based on that that you know sophisticated AI output basically an insurance company on chain right um these kinds of products and services are literally impossible without AI without the ZK that the AI model is high integrity when without manipulation and of course impossible without the rest of the crypto ecosystem to support the the inputs and outputs uh to that service so 2024 is going to be a very weird year in a good way and maybe sometimes in a bad way we'll find out yeah uh I I see you thinking Casey I want to interject here for a minute um in terms of I'm very

focused on security coming from a defi background I think that any industry has been handicapped kept in so far it hasn't figured out security and insurance was instrumental to as far as back as you can think about any technological Revolution required more sophisticated risk management like think about explorers think about you know you you have to have insurance markets to really and crypto struggles that even though smart contracts are highly predictable there are bugs U there are um edge cases where a human may not see it but they're very crypto is a very adversarial environment and these exploits get exploited and things break and they're not fun um and so uh like I invested in a company called um test

machine and they're very much focused on ingesting all the data like that is happening in Defi and all the hacks that have happened and being predictive and being um not only reactive but the hope is that over time you actually become preventative um and can you know if if the exploit is happening across a variety of blocks which is where most exploits happen um then you can at least contain the damage that is being done and I think this ties into what you were saying Casey is is this guard rail like if you're if you're going to make recommendations around the health score of a particular loan on a you want to make sure that the AI is not corrupted or it just goes crazy on you and like um you know I don't know if

that's even possible I'm not technical enough to understand like how you would just have the high degree of certainty that the AI is actually going to be working for you not against you if that's a thing that is thing and Daniel's company will solve this for us in the end in the end that's the goal that's the goal um yeah because the the ver the verification P like I have aund not but like super high degree of certainty that this particular model will uh prevent liquidation for me they will fix my health score will fix my loan to the point where I will not uh be liquidated on chain I mean you can also add um and and by the way this is very

very common with with AI models in production you you literally add guide rails to it you say hey look if the model predicts for example a number below this threshold just ignore it you know let's just keep to this thr and these kinds of like post-processing features we also bring into the ZK circuit uh so you know it's part of the commitment we make to the smart contract and by the way this is the kind of AI output you're going to be seeing and when it's not this kind of AI output we're not going to accept it right that is the the base unit of promise that the ZK proof makes to the program that's on chain yeah also just on this point so Santiago yes I think this is going to be a topic for we'll here on out until we really reach more confidence around

security of models but I'm going to give a call out to my friend Josh Payne who was texting me yes a few days ago and was like looking into llm insurance and I don't think I've ever heard this before until H but this is going to become a thing I think where you basically have an insurance policy around how your llms behave that's one possible I don't want to say it's it's definite but it it it's possible that that's where the industry goes with security because it's such a big risk and because we're dealing with non-determinism for now you might end up creating an entire new market around that so if a company has a chatbot that you're like a travel agent and you make a certain recommendation and all of a sudden it creates a whole problem you get sued because you know it wasn't a

behind that then you might have insurance towards that and so that increases the willingness for customers to adopt AI as a as a product um and have certain guarantees that if things go Haywire in this early beta phase then at least you're covered some maybe yeah that would be I would be really interested in that actually I just love insurance I think it's one of those things that especially in crypto yeah we can talk offline I like an interesting vertical yeah yeah I've been trying to convin insurance companies to insure against crypto risk it's just still not there even though I think the opportunity is great um this has been a fascinating discussion we've been at it for a while um any last topics that you

want to cover um that we haven't covered I mean we could probably make a second segment of just going very deep into the different types of um kind of verticals that we're seeing um but yeah I'm curious if there's anything else that is worth talking about in in the POD at least for now I'm actually just curious just to put it back on you Santiago like you you're putting all the you know Spotlight on us but you also are extremely bright have been in this industry forever and I just would love to hear from your perspective what yeah what's keeping you up at night or what's most motivating to you in this intersection and you alluded to a little bit of it but I would just yeah love to hear more um yeah I've just been perhaps uh well a

lot of things keep me up at but as it relates to AI I'm I'm very excited I mean I'm wearing this sweater like which is effective accelerationism this is the only thing I got for Christmas this is the only thing I wanted uh my parents asked me so I was like yeah I want this because for me I I just don't think that we should uh stop this I think it's incredibly exciting um technology and and um I do think that um there are like governments for better for worse I don't think have a bad intention but crypto at least has always been for me like that for stool like if you think of the three stools of monq of government like judicial executive and what's the other one uh legislative I think crypto is that third like that fourth stool that

really allows civic participation and oversight and like to keep democracies in check because you have an alternative because there there's always that alter that transcends local man-made rules and limitations when you have a decentralized network that supersedes that it's not to say that it blows up the traditional structures but at least it keeps the system in check um whether it's just having more flexibility to go and rent out a GPU in Japan if every year in the US like that that's just like abundant that it just creates more efficiency removes friction from the system so I'm constantly on the lookout of like how can we Implement um AI which is delivering a lot of efficiency gains if you're a developer if you're a artist or whatever and how can we embed that in

crypto which sort of like A Tale of Two Cities it's like crypto is like opening up many different like um possibilities but it has always been handicapped by scalability issues and cost and there's friction there as well so can AI help us like transcend that is um is something that I'm focused on uh but yeah like again to what I liked about modules like the verifiability piece I think is going to be incredibly important like having certainties of like Providence and where the data is coming from so um I'm more excited about that than just like the next decentral ized like a you know compute Network to like you know rent stuff out it feels more enduring to

figure out like the hard problem is what modulus type Company's trying to fix and I'm more excited about meeting those people and figuring that out even though it may take more years but I think that's where the real opportunity will be in my mind I'm very invested that being the future of course obviously but uh I guess I don't since the topic I'm curious uh Casey what keeps you up at night um you know I hope it's not AGI but you know maybe it is um it it's closing the technology it's actually pretty similar to what Santiago is saying it's just the it's like we have one life we're

here let's see what crazy we can do and and in general have I think that I personally believe in Morality and I want to do good in this world and so I feel that this is one of the biggest hurdles we face right now and so I'm I am quite scared about regulation I think there's good intentions in regulation but I think after reading the AI executive order I'm nervous scary yeah yeah I'm definitely nervous and so that's I also I I want to give a little bit of homework if that's okay Santiago or not homework we always give homework in this pod yeah we do okay so I feel that if I'm a really big believer there's I think there's

like two ways to learn generally people can read or people can do and I'm a big doer or Builder and that's how I learn it's just like playing with things and so I think if if anyone listening hasn't done this there's a few Tools in Ai and I'll just do this off the top of my head that if you haven't used just go take the 10 minutes to do it so everybody's probably done chat TBT and yeah if you haven't yes first up go to Mid Journey oh go to Discord get on to Mid journey and generate an image then go to 11 labs and clone your voice in less than a minute okay next go to perplexity and instead of going to Google and ask

anything that would require knowledge that isn't within gp4 and maybe then if you're still interested you could email me and I have more tools for you to use but just even just those four yeah and and that will just like make everything we're saying more tactical I think some people are abstract learners but I think a lot more people are more tactical Learners and so I just want to give those as like starting off jumping off points to go into the world it's a great idea yeah I mean Jad GPT is like I think from a number of users and go to market standpoint it's been the startup that has gathered the most amount of like active users ever or close to being just how many people downloaded this thing

are using it on the daily becomes some people have gone as far as saying it will replace Google as your primary search as your primary way to you know find and acquire knowledge which is I could see that version happening um I wanted to end this uh in maybe a rapid fire but uh I'm debating maybe we should do that uh rapid fire like a whole set of questions sorry have the debate or do the rapid fire uh well maybe maybe not a rapid fire because that might be challenging I may be curious in finding something that you guys don't agree with

that as far as as far as Ai and maybe crypto AI yeah yeah between you guys is there something that in your conversations you have not agreed with um or maybe not just you it could be with someone else maybe within your team Daniel or Casey with your network of people that are at this intersection maybe you because Casey I think you're sit at an interesting Vantage Point around you know core like hardcore AI that don't care about crypto and you also know crypto like crazy people like me that you're at that intersection like what are the things that are the most contentious um Beyond like will AI destroy us or not because I don't want

to get into that discussion I think no one really knows but um I am curious as it relates to crypto and AI what are the things that you disagree with with people the most what's your opinion on model marketplaces uh got it Daniel I actually think we're gonna be on the same but it's a little but they don't work today they don't yeah okay and no one's talking about it and it breaks my brain that no one's talking about it and like people think they work but they really really don't and the reason they don't work is because or at least I'll tell you my thesis I'm I'm curious your thesis so

I'm not going to call any company out but basically like go try to work with the model the incentivized model uh I don't even know what you would call them basically like marketplaces where you're incentivized to improve the model and the problem is that they even perform less than opsource models which should be the floor for them first of all it should that the floor and then you should build off of that with poken incentives yes but they perform worse and the reason I think I'm coming to is there's two options and I don't actually know so maybe you know the first is that no one's actually hosting the best open source models I think that's less likely what I think is more likely is the incentives or like the reward structure

is perverted in a way that it's basically rewarding people to hit benchmarks instead of actual output so basically like someone's getting rewarded for hitting a certain number and that is not a good proxy for output quality yes so what we're seeing is models win that shouldn't win so I'm Cur yeah and I have actually a potential solution that I haven't told anyone yet but I would be curious to discuss with you yeah I'm I'm I'm very curious I think unfortunately to to Santiago's premise I think we're pretty aligned here um as you know I think it's very easy to think of models as these like Commodities that they're like uniformed surface there just little balls and you know you swap them around until you find the best one but often

times at least in production these models the the good ones anyways are like tuned very very carefully they're they're super super specialized and they basically break the moment you give them anything that don't you know they're not supposed to anticipate right this is very different than llms right which are like the first like General list uh uh that can take in all kinds of different inputs and kind of still make sense and so in the context of model marketplaces uh the fact that there's all these like asymmetries and inconsistencies to their surface area makes them not very comp imposable or very tradable or yeah exactly you just like overfit to the index or overfit to the the benchmarks um and then turns out the open source model does better anyways um so yeah it turns out to be a very high friction

Marketplace that doesn't work very well uh I'm very curious how you'd solve it though okay so this might be out of bounds for this conversation what's up good okay I'm listening I think that so obviously VMS Vision language models are kind of this combination of you know vision and language model working in a better way they're not mainstream yet but I think they're going to come out in the next like two quarters and I think basically you would use that as like a gan structure to decide instead of using benchmarks at all because I think benchmarks are too Gable and so you take out the numbers entirely you put like a few VMS that are fighting against each other and then I think that would work I love that the

solution to uh non-performing AI modeling is more more AI models uh I think that's absolutely correct yeah if there was a solution it would probably feel something like that um we'll see okay yeah we'll see sgo I do think this is an answer because I do think we're contrarian in thinking this we are aligned would argue with us yes most people love this idea um yeah like like and so I understand it is and I'm relating it to crypto because a lot of it is through like incentivizing people to do stuff so you're saying there is a set of companies that are incentivizing people to improve the model um and and they're setting it's the problem that you're identifying is it's the benchmarks that are easily GFI and you

see this all the time in crypto too by the way yeah um and and that doesn't it is sub performing to just a pure open source model yes yeah um I think this is a good place as any to end it I I I want to just impart something to me one of the more interesting things I've heard in this discussion was something you said Casey which is in this journey of AI the things that researchers thought would be easy have turned out to be the hardest and the things that we the researchers thought would be hard to solve have become far easier yes and to me that's just mindboggling because it it is very humbling yes I think this journey of AI

has been very humbling to understand what intelligence is and is not um and and yeah it's just like it's pretty pretty crazy I don't it's just one of those we've talked about at length about how little we understand how the brain works um and this is why it's just a fascinating discussion to have and so um yeah I really like that well that's uh that's the appeal of good technology I think right to just be repeatedly humbled uh in in recognizing how little of the world we actually have um you know appropriately modeled um but uh I think it's Santiago you said it you know the only way through is just continue to invest in this stuff and solve the hard problems all out your

sweatshirt yeah I I think it's it's one of those areas where I feel um underexposed um I've been looking at funds because I'm not going to do it myself but I also lean on Casey a ton to find these opportunities and you know really parse through what is and what is not uh to to be fair a lot of stuff that I thought was and crypto ended up being not and a lot of things that I thought were very real it's just that's just the nature of I think came an explosion of interest in things but we'll see um uh Casey and and Dan Daniel was fantastic to host you uh I hope I didn't interject too much to uh you know disrupt the flow of conversation but we'll definitely we

love to have you on to maybe get an update on modulus and also Casey and all that stuff that you're seeing from where you sit which I think is hugely fascinating thank you for having us yeah thank you so much pleasure we we'll call this Empire a crypto AI part one there may be 10 parts so holiday edition as we're all the the holiday edition yes I like that all right everyone thanks so much for listening and Casey and Daniel thanks so much for coming on I really enjoyed this conversation thank you thank you bye sanago hey everyone thank you so much for watching today's episode really hope you enjoyed it we wanted to take a second to just remind REM you about our upcoming digital assets Summit in London March 18th to 20th Santi and I got your

back seats are limited and we hooked you up with a 20% off discount code it is Empire 20 if you heard it earlier in the podcast there's a little competition running at blockworks to see who can drive the most number of tickets so when you register for the digital asset Summit make sure you use our code Empire 20 see you in [Music] London [Music]