Inside AI's GPU Economy & Infrastructure Boom | Sam Hogan, Kuzco

Transcript

95% of all AI compute was used for training and only 5% was used for inference and we're at this kind of interesting moment right now where the pendulum is swinging the other direction the models have become useful and so once you produce the model you actually you know you're going to spend compute to actually make that model do work for you um in the real world to power your applications your your chat Bots your data extraction um all sorts of things all right everyone welcome back to Empire we have a very special guest uh I've known Sam for quite a bit of time uh I'm an investor in Cusco company uh Sam is one of those people that I go to uh my lizard brain to wrap my head around what's going on in the world of AI so Sam welcome on the show thank you

good to be here uh so want we start with just general background who you are what you're building so people know and then that will take us into interesting places yeah so I'm Sam Hogan I'm the founder and CEO of Cusco Cusco is a spot market for AI inference so we match sellers of Idol compute Cycles these are typically uh compute Cycles in higher-end consumer devices and data centers from around the world and we we pair those with developers on the other side who want access to the output of these models um fairly early stage we launched in March uh we have about 5,000 gpus connected to the network at any given time we have a test net right now where people contribute gpus in order to earn points and uh working on kind of what the demand side of the network

looks like and and kind of how developers actually access that comput in meaning meaningful ways um so the the the one sentence version is uh spot market for llm inference got it uh I'm pulling up a tweet here which may or may not be correct but I think we talked about this during breakpoint but there was a shortage of like h100s for a while there and then and then the price obviously spiked and you had the iets of the world really come out under this premise of like oh you know we'll we'll allow for idle capacity and we'll make the marketplace more efficient crypto's really good at that right we're using stable coins transcending you know just connecting and creating Marketplace that it's just more efficient relative to localized markets the price for that was like I think at its peak was like eight

bucks um per hour now it's dipped quite a bit like sub three probably I don't know where it is currently this was like October um so that bottleneck like just kind of like alleviated but like I'm curious how you think about um just observe that observation alone and and and how viable is that like of a business model when you compare to what you guys are doing Cusco yeah so uh this was definitely true there's sort of sort of this like platitude from you know early 2023 that gpus are really scarce and you know everybody needs h100s and that was true at the time Nvidia had only shipped you know a few thousand h100s maybe 10,000 or more but you know not not that many to meet the demand um that we saw

primarily for training so I think at that at that point in time people were mostly talking about these massive training runs you need these really highly centralized parallelized compute clusters where you have you know a th000 10,000 30,000 h100s that are all interconnected using nvidia's Envy link um really high bandwidth so you can you know you can train very quickly and Nvidia did a really good job of kind of selling The Narrative of like hey uh you know if you're a data center everybody's going to want these h100s and if you buy them from us you're going to be able to rent them for like $4 $5 an hour kind of indefinitely into the future and so data centers you know uh highly businesses they took out loans they bought a bunch of Hardware from Nvidia and Nvidia said

great you know we'll take your money now and we'll deliver your h100s in a year and then they delivered everybody's h100s a year later and what you saw it was the price kind of fall for for h100s fall from you know 350 an hour $4 dollar to now we see you know even short-term rentals in the like 180 190 an hour range um per card so so the price definitely has come down you know there's a couple things on the horizon now too that make it look like it will probably come down even further but specifically the h200 and a Blackwell series from Nvidia um are going to continue to put downward pressure on the h100 market Nvidia is kind of running back their same Playbook it seems you know they've shipped some h20s but a lot of their marketing right now is hey you

know uh we might not ship more h20s till next year and they're super scarce and the price the price is really high and then they're they're kind of doing the same thing they go to Data Centers and say hey look you can go you know look if these are super expensive right now you guys could be renting these to to people um so we'll kind of see how that plays out uh I think you know longterm there is going to be a ton of underutilized hardware and there there continues to be underutilized Hardware that's useful for inference um uh and the you know the big training runs you'll probably continue to need the highly clustered h100 h200 uh clusters um uh and that's that's not really the world that we live too much in because we're so focused on inference we can use the older generation cards

just fine so the h100s the l40 s's 490s uh 4090 right now is kind of the most cost effective card to run a lot of the smaller op Source models on so that's kind of where we spend our time let's double click on that you know because people may not really appreciate the difference between training and inference and I know youor on that like let's just like go go go there yeah so training or or pre-training is the process of you know you take a ton of data you take a ton of compute and uh you you crunch you know you crunch all this data using this compute kind of over and over again um and to in order to train a model um the the models that we have today are in the you know hundreds of billions of parameters closed Source models are in the

trillions of parameters these are huge um you know a trillion parameter uh you know full Precision model is something like you know four terabytes um so so like pretty large large models and you know you need a lot of a lot of ram to like load that thing um into a GPU to to inference it so you have the pre-training side side which is where you actually create the model once the model is created um you can you know load it into a much smaller set of gpus smaller cluster in order to perform inference which is you know the the text in text out that you would see from a large language model in like chat GPT or something um so those are kind of the two phases and we think about the compute there's there's phases in between like there's post trainining and

fine-tuning and all that which we can talk about um but largely these are the two categories there's training and inference and we think about them as separate markets um so you you train in order to produce a model and then you uh use inference time compute in order to actually make the model useful in the real world and just tell us more of your vision around like where the real opportunity is you know like yeah different take some people believe you know curious yeah so historically most compute let's say like AI compute broadly gone to training and this is because the the models that we were producing were actually not really that

useful on a on a general scale so you had what you had is researchers using gpus to train models um and you know and then they would you know they' make the model and they'd maybe do some testing on it but the the amount of inference that they would actually perform once the model was produced was fairly small um so you can kind of broadly say you know 95% of all AI compute was used for training and only 5% is used for inference and we're at this kind of interesting moment right now where the pendulum is swinging the other direction so you know now it's probably something like 5050 where half the compute is used for training and half the compute is used for inference um these are kind of rough numbers um and very soon I think in the next couple years we end up in a

place where it's fully swung the other direction and it's 95 or 99% of the compute that's used in AI is spent on inference and 1% is spent on training and that really is just the factor or the thing that's powering that is the models have become useful and so once you produce the model you actually you know you're going to spend compute to actually make that model do work for you um in the real world to power your applications your your chat Bots your data extraction um all sorts of things yeah and you know even like generating new data to train like better models on right there's a lot of inference compute that you need in order to do synthetic data Generation Um so we're kind of like you know let's say a couple years into that pendulum swing and in five years

will'll be on the other side and you know Jensen has like great great kind of Clips on this like you know inference is going to scale a billion times right he said that recently um and and I think it's true I think if you look at the amount of compute that we're using today uh I don't know if you know I don't know that it'll be a billion but several orders of magnitude at the very least over the next couple years so there's a huge opportunity right now if you can build systems that are really good at inference that aggregate compute though otherwise go unutilized yeah maybe for the uninitiated or like less knowledgeable listener but like walk us more through the Cusco thesis here like you're saying hey we'll create a Marketplace there's all this idle capacity we've heard the analogies of

like kind of Airbnb for you know inference you know there's all these gpus around the world not all of them are being used at the same time if there's some idle capacity like there is Idol capacity for cars as there is with you know you know your room and your home that you just like turn that on and monetize um walk us through like um that thesis and what you were seeing kind of maybe the backstory of you founding Cusco yeah so I spent most of 2023 after chat gbt came out building llm applications so you know taking a large language model at the time it was GPT 3.5 and then GPT 4 and and trying to figure out like hey this is this really interesting technology um what are the net new use cases like what does this

enable that you just couldn't build previously because you know you didn't have like there's kind of non-determinism um that llms are really good at and if you know you can kind of build that into applications it enables a bunch of new types of apps that your kind of deterministic programming is is not great at um and so this is you know a lot of the things I already said chat Bots data extraction data normalization um uh and so we spent all 2023 building a bunch of these apps and we got to the end of 2023 and we realized the you know biggest expense for us was our open AI Bill we're spending tons of money on open AI every month and it was actually like sort of getting to the point where it was

hindering things that we wanted to build because just just the cost was too high you couldn't profitably build some of the products that we thought were the most interesting and so I set down this path of maybe we can use open source models to replace the calls that we're making to gp4 and kind of got nerd sniped into this this world of like open source l at the time it was llama 3 or llama 2 um llama 270b there were people that were doing interesting like fine tuning and that sort of thing to improve certain capabilities that it had um and I I wanted to kind of better understand like what the economics of running these models were and why the why open AIS you know apis were so expensive when you could go to together. a and and run

llama 3 or llama 2 and you know pay like a tenth to cost so then I kind of got into the data center world started having conversations with friends that were you know kind of deep in that world and better understanding what their financial you know economic fundamentals are um kind of the cost inputs and outputs where they expect to be able to generate some margin and if you talk to Data Center providers um over the last couple years most of them will say you know hey things are great we're making lots of money and you're like okay awesome like what's your GPU utilization and they're sort of like oh it's awesome you know it's it's it's really good it's like 90 or 95% and you're like oh yeah that is pretty good well what what's the other 5 or 10% going to and they're just like oh

it's underutilized you know we just have gpus that are kind of sitting around because you have this kind of scheduling issue where you know maybe you can rent three weeks of a month but then that last month that last week in the month it sort of sits idle and so we came up with the idea for you know what if we could actually build AI scheduling software that would allow us to take advantage of these underutilized periods across data centers all over the world um and that was kind of the the entry point for Bo scale so we set we set about you know can we build up a software payload that is preemptable which means it can be shut down by the data center at any time um to fill these gaps that exist within a Data Center and we've largely been able to do it um so

we can we can you know work with data centers and depending on how much underutilized capacity they have they can plug our software in and we'll help them maximize their utilization which in turn kind of helps them generate more revenue and we do that without um without you know requiring them to turn off other workloads that they're that they're paying or it's not like it's purely additive right yeah like I remember when I first met you like there's really no reason the value prop is there's really no reason why someone wouldn't want to use this it's like all you know doesn't degrade anything doesn't interfere with anything um maybe for the unex just again like very basic like what like where's the why crypto piece and all of this yeah so when we

first started this I have a a little bit of a background in crypto I got interested in crypto in 2017 you know I think ethereum was trading at like I don't know 70 bucks 90 bucks or something and um I started writing some solidity and was really interested in alternative funding mechanisms for the internet like micro payments that sort of thing and so you know built a couple kind of sample apps and pretty quickly realized like oh this is cool I can send I can send somebody like 20 cents and it only cost me like $6 uh in fees and it takes like 10 minutes to arrive um and so kind of quickly realized like maybe ethereum is not the thing to to use for this and the tech was a little bit early but got reintroduced to salana in 2021 and you know realized hey this is

this is different this is pretty interesting and some of those ideas that I that I you know was excited about previously kind of came back so when we originally started Cusco it wasn't clear that this was going to be a you know crypto project there would be any crypto components and the more data centers that we talked to the more we realized that um a lot of them exist in places outside the US and if you're going to pay them you need a crossb low fee you know instant settlement uh way to do that right that they're sort of comfortable using and so that was that was sort of number one and then number two there's there's this kind of verification piece um that that needs to happen when you're doing when you're building these distributed networks which is how can you verify that the

compute that is being provided on the network is actually what you know what the seller of that compute claims it is um and so there there will be sort of a staking component to what we're doing on the supply side yeah makes sense on the clustering side of things again I'm I'm not like an expert on this so please stop me on my tracks but like my my brain was like I was thinking like is there any degradation by doing distributed like clustering like like like collocation we hear this term like collocation like in performance like you have to have basically in a particular place all of these gpus and so you get more performance out of that maybe that's relevant for training not so much for inference and then you know like you could you could rent like a you know a a

GPU in Europe and then the US and then Asia and like that's fine because for that particular use case you don't necessarily need them to be Co like the cluster doesn't have to be physically in the same spot is that true or like am I totally out of Base yeah no you're pretty close um so so there's all sorts of different configurations depending on what you want to do for training you do I mean this is starting to change but generally you need highly clustered compute uh with you know really high bandwidth between your different gpus um there's people working on kind of solving this problem but that's historically how it's been um and the you know the big players that are training large language models are still using you know these these sort of

setups um you know maybe you can have a couple data centers that are close to each other and you know Google has a facility in in the central us um that's like three data centers with within you know 40 or 50 kilometers of each other and they do some you know they have really high bandwidth connections between the data centers and they do some training there um for inference it's a totally different story um you you don't you don't need these highly clustered machines it really depends on the size of the model but for those smaller models you can actually run a you know fp16 llama 3.1 8B model on a 490 uh which is just 24 gigs of vram and if somebody wants to make a request against that model it's it's just a single request in it gets routed to the

device the device starts streaming the response and that's kind of the entire interaction um there are more complicated setups for inference when you start doing larger models you can Shard the model and do what's called pipeline parallelism where you break apart the weights and then the outputs from each layer are sort of passed to different gpus around the world this is not super cost effective or efficient it's quite slow um so we're primarily focused on models that are you know small enough to F on a kind of you know the largest models that we run can fit on a single h100 node which is eight uh 80 gab cards um and and most of the best models today you know have don't have a problem fitting on that size of machine yeah that's great maybe this is a good

opportunity to kind of transition more into putting some sort of appreciation for like the type of use cases I know we want to go to AI agents which are all the rage but before we go there maybe just give us a sense of like what are these type of applications what are these type of who who are your customers because you guys are live right you know what are these like for what type of use cases are like when you talk about inference like what are we talking about yeah so so we break our use cases into two different categories one is real-time inference which is typically a streaming request where there's a user waiting on the other side this is your you know kind of standard chat gbt type application where a user sending text in and then they're waiting for a response

back um and then you know maybe they send another chat message after that you kind of repeat this process with some some context of what's going on in the conversation um that's one category the apps that use this type of inference are chat applications you know writing applications um uh you know rag pre-processing pipelines where you're maybe doing some data extraction um there's a whole whole swath of real-time use cases and real time is uh is great um but it's it's much harder to serve real time use cases and it is to serve what was that there's like some latency here like if you're if you're using Cusco right like you can't really serve as real time um we we can actually so so we do some like Geographic routing

um to to try to connect you to a you know node that's close to you so the latency is pretty good um and then the second use case is batch processing or asynchronous processing so this is you queue up a bunch of jobs there's nobody waiting on the other side you're just kind of uh you know waiting for a response that comes maybe in the next hour maybe the next like 24 hours um and there's a bunch of different use cases within this that are interesting bunch of verticals a big one is is just data extraction so let's say that you want to you know queue up a million kind of web scraping jobs or data extraction jobs over HTML data to pull out some kind of normalized you know schema from from people or maybe it's real estate

listings or whatever and you're going to run this job once a week um to to get up toate data these these jobs are great for Cusco um because they're not latency sensitive and the reasoning about how you can do resource planning and allocation is much simpler because you have this kind of longer longer term job that's not latency sensitive um so the two big use cases there are uh synthetic data generation for uh fine-tuning and post training and kind of other forms of data extraction yeah how um from a economics perspective like if you're someone like that like a real estate company whatever that is doing this asynchronous you know process like how

the choice of using Cusco or using someone else like what what does that come down to what do you like um do they do you even lead with crypto or is it Hey listen just from a cost standpoint is just cheaper and like you s of like you know have to use stable coin but you know so be it yeah yeah so our our whole pitch is that we are we're more affordable um and we're more affordable without sacrificing the the performance that you would expect from a kind of centralized provider um so we do a lot of clever networking and scheduling on the back end to to make this possible and you know there's there's a lot of like Innovative technology that we've built but I'd say the the primary primary interesting thing is um the sort of innovation on Market structure that

we've done so the data centers that we work with and the providers on our Network are are selling us compute that would otherwise go unused so they would make zero money from this compute right and and so rather than you know them coming to us and expecting us to pay market price we're a buyer of Last Resort that compute is going to expire and so we get to buy it at a steep discount and pass that savings on to our customers um so maybe you can go to a centralized provider like you know together together. AI might charge 20 cents for a model that we charge three cents for wow so you get a X 10x cost reduction just by switching and and to switch it's a you know two-line change in your open a SDK and boom you're

plugged into the network you're saving a ton of money wow how easy is it for your customers to accept stable coins but also pain stable coins I mean a lot of what we've talked about in the podcast is stable coins are a killer use case you know you have the bridges of the world that have just abstracted away like you know like using stable coins yeah how that process has been from your perspective but also any friction that you or resistance that you've gotten from your customers yeah so it it varies widely um you know we have there's a lot of data centers from kind of the the eth2 world that may have a fleet of 30 90s or something um that are very crypto-friendly um willing to accept stable coins they understand kind of the different chains and tradeoffs and and

they're like totally ready to do this and then there's sort of a newer category of data center that maybe a little bit uh less experienced I would say nobody has outright said like we're not we're not willing to work with stable coins or we can't do that um but there's sort of an education step to you know getting people you know comfortable with it um to a certain extent um but yeah it's a we see kind of all all the different takes on on stable coins it definitely feels like it's becoming more accepted and and something that people are familiar with which has been cool to see right uh before we go to AI agents um and just talking generally about what you're seeing in the market and do want to get your take on all the stuff that's going on um a lot of times like I'd love

to understand from a Founder like the Grand Vision like what what is it how big does Cusco get like what is what is it someone like you can be doing a million things and you've decided to purposely like attack this market and opportunity but like yeah what is the long-term vision and how big does Cusco like get yeah so this might be kind of long um yeah so the Grand Vision is is a global market for inference for AI inference um right now we're specifically focused on llms and it's our take that uh llm tokens the text that comes out of llms are is is a new form of digital commodity um and there's

a bunch of reasons for this um uh that that sort of make this true it's it's kind of an emergent phenomenon right now but it looks like over the next 10 years you know llms will become llm tokens will become this digital commodity and we think that they'll trade like any other digital commodity you'll have a market for them you know you'll have like derivatives markets you'll have ways to kind of hedge hedge risk um and we want to be the the global trading venue for llm tokens so in the same way that somebody can buy you know sha 256 as6 Miners and you know they can buy them and they set them up with the proper power and whatever uh they can just plug into this Global Network that is Bitcoin and they know that they will make you know x amount of Bitcoin we

want to do the same thing for AI reference where if you have excess capacity or you have GPU capacity you know that you can just plug that into Cusco and um you know you'll you'll get paid right there'll be some uh some kind of value transfer there um so that that's kind of the global vision we also think that this this sort of uh helps data centers move towards more efficient models um and like business models um by by sort of giving them this fallback layer um that they can plug their Hardware into so it gives it gives them a bit of security when reasoning about like what's the you know minimum amount of money they can make over a certain time period and then it's great for developers because prices come down and

that enables a whole you know class of of applications that may not be possible if the cost of inference is 10x from what we're selling it at right that's interesting and um what would what might get in the way of you guys kind of realizing that yeah so I think the biggest existential risk to us is that you have just massive uh capital investment from you know nation states and other huge players from around the world and they just build these huge facilities and they run them at a loss for a long period of time just trying to kind of win market share and it becomes really difficult for us to compete on price um and and they kind of you know build out the software layer and are

willing to to eat the cost of of uh of running these facilities for a long time and and it you know makes it difficult for us to sort of gain a foothold I think right now we're in an interesting place because if we can kind of get the flywheel of the network that we're building going um then it should sort of be self- sustaining like as more Demand on boards we'll have more Supply on board because they'll be making more money and then then prices come down so more demand joins the network and and you kind of get this virtuous cycle where the the network can grow itself yeah yeah definitely um all right well that's super helpful I'd love to I realize that uh you guys were involved in the salon hackathon for um that agent track to be all the rage like maybe this good

opportunity to transition the conversation now into what you're seeing like in that hole because felt like there's been just an explosion of it uh and interest but yeah maybe just what are your takes there and I appreciate it's a very broad question yeah yeah I mean I think broadly it's really interesting uh there there's a bunch of kind of different threads that I see that could lead to something new um you know right now it feels mostly like exploring the surface area this kind of new design space which I personally love I think that's like a really fun fun place to be um you know there's projects like truth terminal that have kind of uh you know grabbed mainstream attention and for anyone un isue this is the goat

like U right Andy's project and this is go yeah yeah yeah so um I I don't know that I have like a fully formed opinion other than it's really interesting it seems pretty asent I think long term I could see a world where it's sort of like crypto X aai xocial I think like the social aspect of this is what's most interesting if you just have crypto and AI uh probably less so like I think that just becomes sort of maybe an onchain program that has access to a model where it can do some non-deterministic calculations and that's interesting but I think the use cases are fairly limited um and I don't know how autonomous they really become in terms of like making their own decisions and and um or or

even what the sort of like uh like what's what its goal is like you have to do a lot of kind of prompting around like your goal is X or Y and then here's all the kind of different function calls that you can make or different apis that you have access to um so that seems especially early to me the social stuff I think long-term becomes um like the kind of social agents become these autonomous like influencers right uh so the you know the the news research team um they've done some interesting stuff around this they took over the Satan handle on Twitter and uh you know had Satan kind of powered by this llm I think and um uh I can imagine a world where you have a platform like Twitter

or maybe it is Twitter where you have these kind of anonymous accounts or or autonomous accounts that you can go and interact with um that maybe have you know some some uh goal of like maximize entertainment or you know maximize uh value capture and like try to make a bunch of money or or whatever it is you can program them in a bunch of different ways um but broadly yeah I think I think it's super new I think there's a lot a lot of surface area to explore and this really is just the you know the first inning yeah I've be super curious like obviously like I started paying attention with truth terminal yeah and just totally unhinged like some of the stuff that I mean for anyone listening please don't go and you know look up goat I made that mistake that Meme from

like the 90s it's like no but it seemed like um you know Andy who's like the kind of like the father of this like AI agent was like kind of like it trained it a bit but then it kind of like took its life of its own it started to push back like as like a son would push back on the father it was like a really interesting to observe in real time and and some it was just pure kind of entertainment and like it started making money and so as soon as it made a million bucks it was like Hey listen Maybe I'll just sell it all and buy a ranch in Wyoming kind of thing um yeah I I think like the one thing there and you know that was truth terminal was was the first of these that was popular and that

people were interacting with um I don't think that any of the it's hard to separate like what is actually autonomous about this and what is more sort of performance art and maybe you know making the bot say things that you kind of just tell it to say right it's hard it's hard to draw that line you need some way to ify that the inferences were actually performed you know honestly and and that you maybe don't have access to its Keys um which right now I think because it's so early the infrastructure to do that just doesn't really exist and so I've seen a few of these that have gotten really popular and it's pretty clear that it's you know actually just somebody on the background maybe yeah exactly maybe maybe they have like a just like being like just saying

stuff yeah maybe they have like an llm that's going in like responding to comments right away or something right but it's sort of uh it's not fully autonomous I remember when goat made a like it tweeted uh the true terminal account on Twitter tweeted something over the weekend and it had a spelling mistake and everyone was like wait a minute that's not something like an AI agent like would you know I mean like gbt doesn't ever make a spelling mistake um and then and then the you know of course goat like tanks like 50% and then Andy comes back and says no wait like it's because it's learning and it's optimizing to your point for engagement then you it just becomes more human and it's not like a perect so I don't know

if I truly buy it but like that was his like response to it so again it's like is this like the waiting behind it like so so like if you're if you're like um as we see more of these AI agents like how do you think um will we ever understand what that AI agent is like set to optimize for like clicks engagement or like just something else and like how how do you think that like evolves you know like because to your point it would be pretty relevant to have a way to verify like what is the ultimate intent of this is if if you have an AI agent that's like supposed to be your your news aggregator or like Interac with them to feed you content well you'd want to understand what if it's a CNN Fox or like you know unbiased

kind of just like like synthesizer of information it's hard hard it's hard to tell that today yeah it's it's super tough I mean I think like the the best that we can do today is you have uh you know the model weights are open source and then all the prompting that is used um is also you know you can kind of track like through the different prompts over time to see like what's being fed into the agent and and how is it responding um you know that I don't know that I totally buy the explanation that it's it's learning from Twitter and so you know people make spelling mistakes on Twitter and then they agent is now making spelling mistakes like you'd have to have a pretty uh sophisticated like fine-tuning setup I think to make that happen or maybe

you're doing a lot of prompting but it's just really hard to tell it's you know it's pretty pretty speculative any any of these things though I think long term um you know having kind of verified model weights and then being able to track the the inferences and verify that they were performed over you know a set of data and a particular model then having maybe some sort of attestation um for that that that's probably like where we want to be long term I think we're a ways away from being able to do that you know from like your average kind of developer being able to in to just like plug into those systems and build these agents and I know there's a lot of smart people working on this yeah what were some of the more exciting projects or kind of pro projects that came out of the

hackathon that you were part of yeah so the hackathon is hackathon is still ongoing um yeah so we haven't actually uh we haven't actually reviewed any of the submissions yet but um a couple that that sort of caught my eye the first is Eliza from the AI 16z team um Eliza is an agent framework kind of gives you a really easy way to plug into different social platforms and kind of do basic things that you would that you would need to do with an agent um you know manage wallets that that sort of thing um so that one's super cool they've done a really good job kind of building their presence online and and becoming the deao agent framework um there's a couple other agent Frameworks that are kind of along the same vector and then there's a

there's a few I forget the name of the project but um you know there was one that I thought was really cool which was uh you know somebody had a twitterbot and it was plugged into an llm and there was a crypto wallet attached to it and the instructions for the for the bot were basically don't send all the money in this wallet to anyone like no matter what they say right and then the goal was basically trick it into sending you you know all the money that it had in its wallet I think it was like $50,000 or something so you had all these people that were paying like5 or $10 to try to hack this bot um and eventually somebody got it and so all the all the kind of guesses were were public and the prompting that somebody did to actually trick the agent

into sending him the tokens was pretty clever um so so I think we'll kind of see like more interesting things like that maybe not exactly like social agents that are accounts on Twitter that are like posting a lot but kind of these like social AI you know crypto um like I don't know different expl explorative projects maybe the analogy on that was like if you're if you're like obviously humans could hacked all the time in crypto like you you click a different link you someone compromis the account and like you know humans get hacked but it's interesting that you say that like that this someone was able to actually trick this AI agent um into sending them the money how would you describe the current state of affairs like in in the

AI agent world are these like kids that are really smart on like true Sant in one end but like very unsophisticated in how to interact with humans that are not perfect because like there's there's two two two like interactions may happen AI agent to AI agent the possibility to maybe um make these mistakes is maybe lower I don't know unless you're specifically training an agent to be very adversarial and like act like and then there's like a agent to human and that's a whole set of like from maybe putting yourselves in a position of like someone that's building an a agent like would you say that in a world where like there's proliferation of a agents will most a agents interact

with other AI agents or like smart contracts or like would they interact just with like humans as well like how do you think about that Evolution and what to you is like more interesting I guess if you were building an a agent yeah so I think you'll have both you'll have agents that talk to each other and when you say you know when you say agent I think it's like uh you know what is an AI agent right well let's just start there yeah I mean it's I want to know how to yeah yeah yeah I think the answer is like we we don't really totally know yet it's like an llm or you know a program that runs a llm in a loop kind of over and over again with some sort of objective function and some tools that it has access to using maybe it's like a

web browser to get data on something or you know it can post a tweet or browse Twitter um that's a pretty broad definition like you could you could make agents that do all sorts of different things just based on you know those different sets of kind of those uh capabilities um uh so yeah I I don't know that there's a single answer I think that the you know a couple years down the road maybe the most interesting thing will be agents that interact with humans and then interact with other agents to accomplish some goal given to it you know by a human I think these social agents right now they're they're fairly Limited in what they're actually like capable of doing you know they they're not doing like meaningful work

it it really just feels like it's entertainment hopefully that will change at some point and you have you know you can run a whole business and it's just you and a bunch of agents that do work for you but I think ways away from that do you think it's important when humans are interacting with an a agent that they know it's an agent that's a good question yeah I mean I think the the you know 2020 version of this is when you're interacting with people online is it important that you know know that it's you know a Twitter bot versus like a human I think important yes um is it totally possible is it going to be totally possible probably not to to actually you know like you need some

sort of identity layer right something like World coin potentially to do this kind of verification of of human yeah this yeah I go back and forth on it because clearly like when you're interacting like you invert you say I'm interacting with a human right now well you Sam you've been trained at a bunch of you personally have absorbed and your principles and values been shaped in a way that is very I don't know all the context of that now I might know you and I might Place some sort of judgment on you but it's not like true open source like we're asking to your point on atation understanding all the weights and how this a has been trained I do appreciate that there's a possibility to do so like why wouldn't you but when you compare with the interactions that you

have with news and where you you get your information like you have zero understanding right most people under the impression that you're very connected but and so therefore informed and I would challenge that presumption say we're very connected but we're not necessarily more informed because we're living under this illusion where like news is an example it's totally broken right you have to have someone buy Twitter and like really T like break that otherwise like there's all this censorship and all like you know if you're listening to Fox and then you get retargeted and your feed on Instagram is very much like your version of the world that is to totally different from someone else's and that's a virtue of how the algorithms have been very much um constructed to optimize for one thing

making more engagement and therefore more marketing dollars and we don't that but this is the State of Affairs yeah so is to to like assume that like the the a agent interaction should be perfect and transparent you understand all the weights I mean maybe that's like the upshot of like why a like the bullish optimistic case of AI agents you know yeah so you know again the 2020 version of of um what you're talking about is just like the Facebook you know algorithmic Facebook Newsfeed that kind of learns what your preferences are and it just shows you the things that it knows will keep you engaged you are being kind of subtly manipulated or even not so subtly manipulated in various ways right because it's objective the objective function of the AI that

Facebook is trained to run that Newsfeed is maximize engagement and so it takes kind of all of your historical context and it's it's this personalized model that is that can keep you engaged for longer than you would otherwise you know be engaged and it it changes it's designed to change your behavior in a way that is good for Facebook and it's not really designed to be bad for you necessarily best maybe I don't know yeah it's not like you know I don't think Facebook has like uh is trying to like intentionally mislead people it's just trying to maximize profit and if that comes at your expense then like well you know that that's a bummer for you I guess um but but what AI what AI allows us to do and it sort of lowers the barrier for

building these systems that can be personalized to an individual so rather than needing the resources of a company like Facebook um to to build personalized models and personalized you know feeds and all that llms in particular lower the barrier to entry and make it so that you know a random Dev like the guy who built TR truth terminal can kind of build these systems that that uh change depending on who they're talking to kind of change their own objective or same objective function but sort of change you know how how it's going to get there um so I think that's cool it democratizes some of this Tech and then the other thing it does is it it allows people to build um you know technology that pushes back on these systems right

so maybe I have a an AI model that is that I train and that you know I get to put my own objective function into which is like I only want to see you know really cool AI content on Twitter and I don't want to see any of this other news what's going on around the world anything that's distracting um I just want to see like these handful of things and then it acts as a filter for me so now you know not only does you lower the barrier to entry to build these systems that are potentially dangerous but you you lower the barrier to entry to Building Systems that protect you from these other dangerous systems so I think it becomes fairly adversarial where you probably have like you know again it's like agent to agent communication with humans on either side that have given

the you know their their AI an objective function of some sort um I think we're a ways away from this being reality but but probably maybe not as far as we would think yeah um two questions but um the first one is on on training when you say filter say in the example you want to filter content AI how do you um get a sense for like when we talk about like training a lot of the data set has bias um has you know just just the nature of the Beast right um unless it's like block data could be pristine could be like tamper but like all the other stuff that you're training it has bias has sort of like human touch and footprint there like how do

you how do you tease out that and it's just not possible or how do you kind of are there methods or techniques that allow you to you know kind of break from from that or at least identify it yeah I so I'm I'm not a a you know expert on training by any means um I'm sure there are people that are developing techniques to do this sort of thing um I think having like a for open source models in particular knowing what data they're trained on I think is going to be important long term again I don't know how feasible it's going to be um but I think equally valuable what is being trained on and what is excluded like Gemini for instance if you ask him what the perfect diet is it probably say no meat because it's all woke people that have trained this model you

know it's like yeah totally and I think like you know at the end of the day like there's no like you know it's like like okay design the perfect training data set right that has no bias well it's just impossible to do because bias means something different to every person and everybody has kind of their own world viiew so there's there's no like way to revert to the mean in in this case right um you're you're sort of like at the whims of not only the people that are curating the data but also what data is available right like if you have uh if you have a let's just say some African language that only like you and your tribe speak and none of it's on the internet well none of that data is going to be

available to the lm's not going to understand your language um is that really bias or is it just kind of like oh the data is not there um yeah you're training any time you're training on human data humans are not perfect so therefore there will always be quote unquote bias because it's all subjective so someone will always be able to identify it like it's not perfect so there's bias yeah yeah and I think like you know we we talk about like AGI I think the goal the goal posts on what AGI is have changed quite a bit over the years like 10 years ago it was could it pass the Turning test could it full full fool a human to thinking that it's another human we passed that point you know a couple years ago with probably gpt3 um now we have models that are

substantially better than that and there's a lot of people that are like well we don't have AGI yet and um it's like yeah we might not have the super intelligence Godlike AI that is smarter than every human at every every task but we have ai that's very very useful um for for different things and so kind of finding like okay we don't have a you don't we don't have ASI artificial super intelligence but we do have this thing that's pretty useful how can I make it useful for me even if it does have bias and it's not perfect and it's going to get things wrong um you can kind of take each of those you know undesirable qualities and and build safeguards that are specific to you know what what you want going back to one thing that I think I've I've seen you tweet a lot is

you and others is this idea that like you will see companies that have started by one Dev like that just reach massive scale because that's what AI allows you to do whether it's co-pilot and then many other things after that that just really Empower like one individual and like just you're able to create so much more more productive but also have more scale um and I guess does that I want to get your take on that like does that concern you is this sort of like a Frankenstein kind of like Matt scientist and then built this machine and like um or or like you get excited by that which is hey now you don't a need a lot of resources and B the impact that a particular individual can have like

you're able to see these companies out and like have a lot of impact much faster in that timeline compresses yeah I think this is a natural progression of Technology if you look at the history of you know humankind over the last few hundred years as technology really started to proliferate it's sort of this history of democratizing access and you know giving giving power to individuals through through you know tools that we've developed so it used to be like all you know there were like these massive companies every company was had tens of thousands of employees and then you know like that that was most companies that were you know like meaningful impactful companies then you had a lot of small businesses that were maybe restaurants and stuff but nationally there were just these like very large companies um

because you needed a lot of human you know human labor human capital to to do anything and we've over the last really the last 50 years we've just started automating different different parts of that and building tools that give us leverage so you see companies getting smaller and smaller and you know I mean examples from even the last two decades like I think WhatsApp was like 54 people and it got acquired by Facebook for you know 50 billion and it was you know had this massive worldwide application that was used Everywhere by by tons and tons of people um so you had this really kind of small high performance group of people bu an application that could be used worldwide and have really you know probably largely positive impact for people um and so now ai I think is just

maybe the internet was like a big speed up there because it allowed anybody to connect to anybody else in the world and now you know AI is sort of like the next layer of that the next layer of like efficiency gain and and leverage that that we'll get and so um yeah it's very likely that we will see really small teams doing you know really outsized numbers or I'm in San Francisco there's you know I know of a couple like two and three person teams that are doing 5 million ARR and the you know they started a year ago um so that's that's cool to see I think you know empowering people kind of lowering the barrier to entry to building interesting Tech that is high impact is probably a good thing um there's obviously some downsides but

I think it's you know net net it's a positive and I I don't think there's any way to stop it and it feels like it's really just a continuation of the curve we've been on for a while yeah definitely I I go back to what you said earlier we is in a world where there's always going to be bias and training if you have one AI agent that's really skewed the waiting is really off or like malicious intent or whatever when someone because the barrier is so low someone can actually just launch another a agent that counteracts that and just brings at least like if technology always introduces more competition especially if it's open source and therefore that that's like the only way that we can that we might be able to push back versus you don't want to be in a state of the world where it's just

Facebook and that's where you get your information and there there's no ability you know world where you can have multiple agents like competing against each other with different weightings and different intents then at least you're able to expose the truth or the lack there of by comparing these two right and so that that's a better version of the world than in what we are now in a very highly like centralized monopolist like with monopolies that just like you can't really like compare yeah and and I think you know the interesting thing about that or one interesting thing is the best open source models right now are actually produced by Facebook right yeah right yeah um and if we didn't have those if Facebook wasn't publishing those then we would kind of be in the situation that you just described which is you have you know Facebook has their

llama apis open AI has their you know G GPT apis you know mrol has their thing anthropic has their thing but it would pretty be be pretty limited to like four or five you know maybe 10 companies in the world that have access to this technology and we still might end up in that situation right Facebook could decide hey llama 4 is actually going to be closed Source it's too powerful whatever their reasoning is um uh so yeah I mean I think we need these sort of Grassroots efforts to to build AI that uh isn't biased and isn't going to have to answer to its its creators or you know have face the risk of being like taken offline at any time yeah I think I heard it from Kai Fu Le uh he said like this whole quest of

understanding you know Ai and AGI will really allow us to understand what human intelligence is and what it it is not and I think the latter part is actually quite interesting because even though in a world where we might truly get to like super god AI kind of like State we're still gonna learn so much about what human intelligence like that process of it's almost like a process of introspection in many ways yeah yeah I was curious to get your your thoughts around that and also this whole idea of like what is human value like like as I was like hearing you say this I'm like in a world like maybe many years ago like no taking was like like a skill right and now you can use stuff like granola and like it's great not taking and so like it really shifts like what

do you think is a core will be the Core Essence of like human skill like skill set like what are the thing what is the thing that like will be truly valuable in this world where we can like delegate and Outsource all most things to AI yeah so I there's sort of like two parts to this question you know humans are are still very like physical right like we have physiology that and like the way that we feel impacts how we think impacts what we do um you know we have this this I think really important often overlooked physical side of what it means to like be a person um and um so I think that you know we'll probably see

kind of a doubling down on like inperson experiences uh more more broadly like you know like I don't know uh the just one kind of small data point like gen Z for example uh everybody's taking dance classes right like the the younger generation if you're born after 2000 the uh enrollment in dance classes for people under the age of 20 is like 10x what it was a decade ago right right uh and like dancing on Tik Tok is kind of this like generational thing there's like all these people that just want to move and like and it's this generation that grew up so online and you can kind of see this like push away from that and like this push towards like I want to be in the real world I want to hang out with my friends I want to dance I want to

move um so so I think that we'll you know as we continue to build the like online AGI God using llms and whatever else uh we'll kind of simultaneously have this other track of like in-person experiences things that really only humans can do um will probably start to put you know more more effort into those things I mean even even something like rfks you know make America healthy again right sort of this like recognition of well we've actually been unhealthy for a long time we've probably been spending too much time being sedentary not moving around and we need this course correction back to like hey let's take care of our bodies right we're also overstimulated on uh what we see online intellectually and otherwise um that

we've sort of ignored this like other other part of us and this sort of like natural intelligence that I think comes with having a physical physical presence um so yeah I don't know that's kind of some high Lev thoughts I don't I haven't spent a ton of time thinking about this but I think it is interesting I do uh do like something you said there which is this this like movement to go back to at the end of the day we are very social species and Technology allows us to coordinate better but at the end of it it's like technology is valuable because it it it it sort of addresses that need to be social and feel validated and and that's a very social interaction and so like I love technology for that reason for the coordination aspect and I think crypto is that like crypto becomes Central to that

because it allows us just to coate better as species and AI fits into that equation really nicely think about like when when um when the Apple watch was launched like everyone in the watch industry said oh you know good luck Rolex PCH AP all these like watch manufacturers like you're done you're dead and it was quite the opposite like the demand for vintage watches and and traditional like analog not digital watches just went up dramatically because just nothing will replace that in fact the your craving for that nostalgic like like human craft uh is super powerful and maybe it's because it evokes you know like tradition or just something very human to it and so I think like in many ways like I would I would be if I from an investing

standpoint I'm I'm going to look for things that like tap into that nerve that yeah will only grow more as we become as this other part becomes totally abstracted and Technology super abundant so I the hope for me the upshot is that we go back to like ancient Greek times where people were just paid to think we solv like we solved this issue of like having to put food on the table maybe it's Universal basic income but like in a world war technology is super abundant you free yourself from like the bottom tiers of the mass low hierarchy of needs and then life becomes great because we're truly social species and then we can I think that's like the my my my positive up like case for what Ai and crypto can do um yeah yeah I mean

that that's pretty interesting I think like you know humans will always have ways of building uh social hierarchies right like you know it used to be like you're the the best hunter and so you kind of exist at like the top of the social hierarchy and now it's like you're the best you know mem coin meem coin sniper and so like you're in your telegram chat everybody's like this guy's a God you know and so I think I think those things like change over time and if you cut off the bottom half of Mel's hierarchy you know it's not like uh the desire to accumulate resources and exist higher up and this hierarchy is going to go away we just have we have different things that um that we use to kind of build those those hierarchies right to compare to compare each other effectively um

well always because we're social and we want to feel validated underneath all that is is this need to differentiate yeah yeah totally which which you know is funny because there's a sort of like uh um I think maybe like naive tendency over the last couple years in Western societies to to sort of like why can't we all just hold hands and and like Kumbaya and like you know everything just be you know one giant world government and everyone gets along and there's no conflict and and um it's like so antithetical to human nature yeah oh yeah there's a great book uh called the near nearly perfect happy people which is this uh we talk about like Scandinavian countries being the happiest places in the world and this American is married to this Danish um

this Danish girl and he goes and lives out there and he and he documents the entire evolution of because of immigration these countries like accepted a bunch of refugees and and the composition of the society was when we talk about perfect was super homogeneous it was like all Vikings basically and as and as it became more heterogeneous there was a lot of tension that you see in other places and that's just you know um like like colonialism like Europeans just drew lines in a map in Africa and that created a whole hundred plus years of conflict you shouldn't put the hudo and the tosis in the same place yeah Don't Force It that's okay we should celebrate our differences but I think that point you make is so true

which is not because you celebrating differences doesn't mean that you can't accept them and somehow like this woke culture is like oh we should all like be you know the same and that's like not the way the humans are designed to operate it's uh we should celebrate our differences not try to become one yeah and I think there you know there will be this sort of like convergence like in a 100 years you know will be this effectively you know entirely globalized soci Society right like you know uh even the way that people look will probably Converge on you know kind of one thing well yeah I mean if you just imagine that people start moving around the globe um and and we don't have the same kind of geographic um you know constraints that we used to or it was just hard to go to

different places you know over time it seems like humans will probably Converge on you know looking looking similar um uh kind of cut off the ends of the bell curve a little bit um so yeah I mean it'll be interesting to see how how all that plays out over time and kind of how AI like fits into uh how we build new hierarchies and like what are like like okay so you know I think this Tech the technology that we have like even that we've had over the last couple decades you there's this uh potential for sort of like getting lost in it right where like you just end up spending all day on Twitter spend all day on Facebook and you sort of neglect the other parts of like who you are you neglect like your

physical whatever it is and um you know maybe that makes it hard for you to like find a partner and like reproduce and and whatever and and so it's sort of like as humans we have to figure out how to build safeguards against some of this technology and we've always had to do this you know it used to be that um Everybody farmed and then we had this big Industrial Revolution and so a bunch of the people that were farming you know farming became much more efficient so you needed fewer farmers and so the people that had been Farmers for a long time had to figure F out how to adapt and a lot of them moved into cities and became you know they worked in manufacturing plants and that sort of thing and we have versions of this today that are you know not not quite the same

but I think you know one thing that I worry about with AI is is the the guy who just like spends all day talking to Ai and he has this AI that kind of like learns his you know preferences and and he starts to sort of neglect what it means to be a person and and becomes very like over intellectualized and um and you know and it effectively you know splinters off from society to just exist in like his own little bubble with with all these AIS right sort of replace all of your social interaction with this llm that that knows who you are and knows how to like you know cater to your needs and keep you happy um so we'll see as you were saying that it reminded me of

sidartha uh the tale Buddha like he lived in the enclosed castle and for his entire life and he never was exposed to suffering or just a human condition it was just like a perfect existence um and then as soon as he he walked down the street and saw a leper it just totally changed his perception and he felt pain he felt suffering but like that in that Journey like it it humanized him and then ultimately reached inl men so um I think there's always going to be an element of Being Human and to be human at to ER like we are social species or are mistakes we're not perfect and that that's what makes it you know interesting I guess um maybe maybe as a parting question to is um and I've been asked this so I'm really curious to to get your thoughts on it is we talk about

immortality a lot that's something that um you know people are now we thinking more and more about yeah if if you had the possibility to trade five 10 20 years of your life to put your brain as an AI agent of sorts and live forever um to have impact to interact with everyone but you lose your your body sort of you're done would you would you would you make that trade I personally would not yeah I I I uh I I think it's like humans are always fighting against death right and so the idea of being Immortal is like so appealing to to a lot of people um but I think death is like a big part of life and and like to sort of thwart that is

you know maybe Noble but but to have to give up you know a large portion of my sort of physical life um to exist forever I don't know that just doesn't seem that appealing to me yeah it doesn't me either to be honest yeah I'd rather live quality like Health span not lifespan is what I'm optimizing yeah totally yeah hope hopefully we uh continue to make that differentiation for sure Sam this has been a fascinating discussion um I I I'm really excited what you guys are building obviously and uh people should go follow you on Twitter because I think you have some of the more interesting takes on AI and that you know someone that doesn't truly appreciate it on the ground level like I go to you and your account just understand what's going on

um any any parting thoughts I mean it's been a great discussion I think people really enjoy it what you're building ausco but also your general takes on AI but anything else that you wanted to cover yeah I mean I spend a lot of time thinking about Ai and and less time thinking about um crypto uh is I think the AI side of what we're doing is is you know kind of more interesting um uh but yeah I mean I would say you know if you're if you're coming from a crypto background and you've been kind of deep in that for a while the AI stuff is is really cool and Beyond just the AI agent crypto piece you know I would encourage people to go and if they're trying to upskill or figure out how AI is going to impact the world or impact their lives

like go try a bunch of AI tools go figure out what works for you um pretty pretty nent design space so there's a lot of cool things being built and figure out ways to get involved and and to incorporate AI into into your life um I think that's probably the you know the the big skill to learn over the next couple years is how to you know find AI tools or build them yourself and and uh put them to use in your own way it's uh it's always funny to go from the like technical to the like you know kind of more like soft human parts um as about but it's good I think I think that stuff is even more interesting than a lot of the technical stuff frankly I think so I mean you could you could go on and on

about like the all these things candid you don't no one has a perfect answer to it but it would be really interesting to like look back on this like six months or in a year or two or five and be like God what was I thinking you know it's like Terri wrong and I think that's that'll be a good thing yeah totally I also have like some kind of negative takes on on like Ai and AI girlfriends and like getting stuck into the machine and I'm kind of like I don't know how much of that I want to be super like public about quite yet so kind of that's fair actually I'm actually quite bullish on it uh but like as a way to eventually graduate into the real world so to speak yeah yeah that's interesting yeah yeah I think um I like I want those tools to win but

I'm not sure that those are going to be the AI tools that actually do do win the like the dating coach I think is kind of what you're talking about right go follow Cusco go follow Sam uh we'll put those links in the show notes but uh Sam thanks for much for coming on man I really enjoyed this conversation yeah yeah likewise [Music]