Brian Behlendorf & Tricia Wang: Shadow IT to Shadow Agents | Intelligence at the Frontier 2026

Transcript

[music] This is the session that many people have been waiting for. We have the Brian Bellandorf here and we have him and his colleagues to thank for having open source. Open source is so big now that I just got a call from the UN that they have open source week that I didn't even know about during June 22nd. Um they think that, you know, open source is critical for SCGs, for humanity. Um, and so I think this is a really important moment as we're talking about funding the commons. As I was talking to David about this and why advanced AI society is partnering uh with intelligence at the frontiers is that a big part of securing our commons is to think through security and think through what are the things we have to keep common and

there's a big movement now but we don't have a name for it. And I thought well why don't we bring to stage someone who has successfully named a movement uh open source and let's hear his story of how did he do that but and then more importantly what are the lessons for the time now in terms of this new movement where I need to we need to have named which is proof of control um being able to say you know for our agent ticket orchestration for uh scaling AI we need to have security measures and so we're going to have a great conversation with Brian Bellorf um he co-founded the back Apache foundation which is amazing and then also co-founder of OSI initiative.

So we're going to talk a bit about that but you know Brian's got this history in open source world and then he's got also the history in the decentralized world having running hyperledger so I can't imagine a better um person to be interviewing and also adviser for advanced AI society. So let's bring Brian up to the stage. I think that mic works. Please give him a big round of applause. So we have such a full audience. Thanks for sharing who's has been waiting to come to your talk. So >> is it the word? Okay, works. Great. >> We're on um >> It's funny you said the Brian Bellorf. There actually is another Brian Bellorf out there. What's most embarrassing is he has a much better GitHub repo or GitHub history contributor history than I do. So, but yeah, that's >> but he did not take sorry.

>> Well, to us you're you're the to many of us in the room. You know, we have tour project in the room still. We've got inner archives. I mean, like so much is built off of, you know, a lot of what the community um you started. So I'm curious um from your perspective we just heard from Jeff Bernstein talking about the importance of naming something naming a movement and we think oftentimes we just have to build a technology right and then it's going to take off um and before we get into you know just thinking talking about proof of control web 3 I just wanted to get your thoughts on what was your work in what were the what were you and your colleagues thinking at the time in terms of naming open source it was like it was a term that was already out there but

what was your thinking around that and your process around that? Yeah. So, let's turn back the clock 30 years, whatever. Um, and I, um, the internet was still pretty young. The number of people online was still pretty small. In fact, the tech sector as a whole was still a hundth of what it is today, right? Um, and the software that was driving the internet before companies like Netscaper started, uh, before uh, Microsoft did, you know, Windows 95, which officially supported the internet protocol for the first time. A lot of that software to connect these pieces was uh open- source software. It just wasn't called open source software. It wasn't even at that time called free software although the free software foundation started I believe it was 868

' 87 that f time frame but it was academic research software built by the people defining the protocols online right that the TCP IP DNS SMTP as like proofs of concept that also happen to run well enough to run in production right and the the whole air of it was you know here's the software it'll help you get online it'll help us build one internet Um it came from the government. >> No, no. I I mean the government helped provide some critical funding. Al Gore even can take some credit for like having helped leads a lot of the funding for this but a lot of it was from uh places like uh the well the original ARPANET was perhaps government played like a convening role in that um and

saying this might be a good idea but it was researchers uh all all over the US all over the world really uh places uh like ISI uh at at USC um uh places like Stanford and Berkeley uh but it was the researchers writing papers uh publishing you know protocols and then inter you know interconnecting their systems and and and forming kind of ad hoc lightweight organizations like the IETF which was where they would get together in a radically transparent way and kind of define here's how this stuff should be built. Does somebody have a better idea? There was like this essential humility to it.

>> Was it still the time of BBS's? >> Well BBS's you were popular in the 80s and and and and 90s and I was on Prodigy which is like an IBM BBS. There was CompuServe, even AOL was getting started. And I think for a lot of people, their first experience online was in a walled garden like that. But I remember starting as a freshman at Berkeley in '91 going, you know, falling down this rabbit hole of getting an email address and then finding out there was an FTP site somewhere with interesting software, finding out what Usenet was. Does anyone remember Usenet? I think great. Those who have never heard of Usenet are doomed to reinvent it over and over. Um because it was this distributed Reddit basically. Uh and it was kind of awesome. So I uh so that era right of the early internet

technologists the people who were the early users was very much steeped in this the fastest most efficient way for all of us to build something bigger and better was to give it away to be transparent about our work to be cons uh in fact there was a what was it that John Postella came up with this principle somebody might correct me if I'm wrong um be conservative in what you send but liberal in what you accept in terms of tolerance on the network right so all these kind of lowlying or principles Another one that's very much a part of Unix's philosophy. It was small pieces loosely joined, right? So these kind of it wasn't even like a constitution. There was nothing formal to it, but this informal era where if we could work together to build something bigger, all of us would be able to pay

rent. All of us would be able to have a comfortable life as researchers, but also um working now in companies that were connecting online. Uh and we didn't have to own the center of the universe in order to do that. We didn't have to be at the heart of it. So um uh uh now moving on in time a little bit right as the web came up came up as uh uh the first web browsers which were again research projects the NCSA NCSA mosaic I don't know if anyone here is old enough to have also used it but like okay um you know there was that carried that tradition kind of forward um uh but it was also now a moment when the um the tech sector was starting to take the

internet seriously with the public was starting to take the internet seriously and start asking questions about, you know, who runs this, how is it governed, that kind of thing. And in that moment, there was I remember being on a mailing list of other technology developers who we were building on the NCSA web server. I was at Wired magazine. I was at a couple other companies setting these things up. um and I uh we were just sharing patches with each other uh that were improving this underlying technology and then uh but feeding it upstream to the student developers at the University of Illinois who were um uh kind of acting as kind of the the the the coordinators for releases, right?

And then they sent an email to this list, this email list that we were all on um saying, "Hey, sorry guys, but all of us are quitting our our student kind of careers and joining this new company called Netscape uh to go commercialize the web." And we were like, great, that's cool. Like, we hope the best for you. Um, but we're kind of happy using this thing here, right? Uh, and we'll just continue improving it and once Netscape server is better, we'll switch to that. So, there's this like pragmatism at the core. Um, but we were also pretty happy with what we had. Um, and but there was still this idealistic tinge too, which is in 1995, if you looked at the desktop world, it was 95% Microsoft Windows. Mac was there. There was it was like 5% of the market of

computer users were on on uh on on a Mac but you couldn't really say that there were open standards for the desktop it was one company or another and we were worried the web would become like that >> right in that moment was there awareness that we're in the browser wars like is that >> not I think there was a sense that there could be a war that wasn't just a browser war it would be an internet war where building websites uh and sharing information would become like Mac versus PC where if you wanted to have a meaningful presence, you'd have to do it twice, right? Uh you'd have uh your website that worked for Microsoft and your website that worked for Netscape, right? Um and I, you know, writing a web browser is 100 times harder than writing a web server. So I didn't think many of

us thought that like I I there would be open source web browsers. Like that almost was like not anything that I wanted to take on personally. I just wanted to keep building sites and I wanted to know that I didn't have to do it for a number of times for a number of browsers. So I cared a lot about standards because standards had built the internet standards whatever uh is why we were able to get to where we were. Um and so we felt and and we being this group of other uh developers improving this NCSA code said why don't for why don't we organize this better not only for pragmatic reasons so we could use this but maybe in our hearts of hearts I hope that um we do play at least a small role in keeping the internet from being reproprietarized >> a small role

>> and and again I think humility is a part of this and I and I and I want to emphasize that's something that I think is still important today you know I mean just flat you you mentioned you you can't mention web 3 now and a lot of how you talk about um the work at proof of control part of it is I feel like in the the crypto space and I say that having been more adjacent to it those uh on on like the board of the filecoin foundation and other things but I think a lot of people tuned it out or got burned by it by what felt like overinflated expectations that that came about in that era >> right and and actually saying you know anyone who works on centralization is negative is bad you don't really understand society >> or or just that you So I I will take

over or that there's yeah this inexorable push that obviously every app is going to be a web 3 app in the future right um other things but but my my key thing is that that humility at the core was what the was what was in the environment when a group of folks said um we need a term that describes this phenomenon that's more than just what the Apache software foundation which eventually it became a nonprofit around 98 um but what these Apache developers are doing what these Linux developers are doing because the free software thing. While a lot of us conceded it was really important as a because it kind of framed software development as a an access to the underlying source code and the products you use as a human right.

Something that I remember looking at back then and going this is going to be much more relevant in 30 years than than today when software is inside every doorork knob or every car or every light bulb. Yeah, your ability to have agency over the world around you is probably going to be pretty tied to having access to the source code inside those systems. But that was sci-fi in 1995. At that time, it felt a little bit more like if we could just find ways for companies to be more comfortable when we build websites for them and we tell them it's built on open on on free software, right, that they can support themselves, it it was it was hard to do. Um, and there was this moment that we talked about, right, where as Apache started get to get more widely used when um, and

as other open source software was starting to mature, things like the the LAMP stack, if anyone remembers that, started to actually pose a credible threat to enterprise software that companies like IBM and Microsoft would charge comp, you know, enterprises six, seven figures for. um when we felt like we needed a way to in uh up uplevel the respectability of this communitydriven effort and demonstrate that it's not actually at conflict with uh with um making money, right? Because a lot of us were wanted to make money in this world and to some degree the free software movement felt a little bit contrary to being able to build a company.

>> It was anti- capitalist also. That's how it was framed. >> Right. Right. And a few years after we rebranded our what we were doing as the Apache web server and it was getting out there and being picked up very widely and we had the benefit of good timing. We had the benefit of like everybody trying to get online uh within a shortened amount of time 95 to 98 99 um and and and and the really lucked out in having some really good other developers joining the project and making it fast supporting thousands of websites on a single PC that kind of thing. Um, and and what was also really lucky was that when you deployed a web server, hidden in the headers of a response was a an identifier that told you what software

was running underneath, right? So you could access www.fbi.gov and hidden underneath it, you'd see server colon Apache, right? Um, and and so at one point you could, and in fact, there was a company called Netcraft that emerged. They're still around. They still do this web server survey where once a month they scan the entire web and you know send it a just a basic request and try to see what software is running behind it. And that was a benchmark that allowed uh suddenly allowed us to see not only are a lot of people we think are using it actually using it but like 60% 80 I think the highest it got to was 75% of the web was running on top of the Apache web server.

So there was this sense that this is big. Another data point that emerged in conversations, private conversations we had with some folks at IBM uh where they had asked their top uh hundred I believe it was uh enterprises like the CIOS how many of you are using free software in your enterprises right like Linux and or Apache or these others and like only 30% of the CIOS said that they were using it. Um, but then they asked like three or four levels down like to the systems administrators or IT managers, how many of you are using open- source software? And like 99% of them said that they were cuz every one of them had it for a DNS server or print server or something like that. Um, uh, and that was when they realized there

was this shadow IT uh, kind of phenomena. this this um uh this existence of open source that the CIOS never saw because there was never a purchase order for Linux, a purchase order for Apache or any thing that they had to do >> and had you defined open source by the time of the study. Can you just give us a history of the time? >> Yeah. So, um in in 1998 there was those two two kind of pivot points. one specific to Apache, which was when IBM showed up, this ragtag group of developers, you know, with their own small businesses was like, could we accommodate an IBM showing up and being a part of this community? And we kind of said, yeah, if they followed the same rules that the rest of us follow with in terms of working publicly, submitting

pull requests, you know, uh following the Apache license, that kind of thing. Um uh and and so that that happened uh kind of uh Marchish I think of '98 and then uh in the kind of June 98 time frame is when um there was a kind of a meeting of a bunch of people from different open source project or different free software projects with different licenses often many of them not the GPL going is there a way to elevate public awareness of this and this was partly convened by Christine Peterson from the foresight institute so I want to give credit to foresight for having had the foresight to be at the center of a lot of emerging stuff uh uh recently. Um as well as Tim O'Reilly. So Tim O'Reilly also had this instinct that

there was some phenomenon going on that really just needed a way to be harmonized, a way for people to discover each other and realize that they had a lot of the same interests at heart, right? And so that I I so Christine convened this this small group. Uh it was about five people and like we should come up with a new term. Christine suggested open source as the term. uh a bunch of us said that makes sense and so what was but what is it attached to right and in the Debian project the Debian free software guidelines had long existed as a way to qualify what third party software should be included in a distribution of the Debian uh Linux operating system right uh and they were a reasonable set of of things that

actually were about conferring agency to individuals you have the right to redistribute the right to make modifications uh uh the uh um you a set of about nine principles by which that you could evaluate any copyright license uh in a very objective way. A way that lawyers can generally all come to agreement like okay this makes sense the legal defensibility it already >> that was that was when it was it was I mean it was like a thread that connected a lot of these different related things. So part of it was about marketing and building a level of respectability in the sense that you could use this code and it wasn't anti- capitalist at its heart. It wasn't uh you didn't have to you you know buy into what all the free software foundation uh uh principles were or or what Stallman would say and

yet it was a path to that right it was also a legal kind of framing because so many companies are riskaverse so they had to understand can they use this stuff without worrying about somebody else's IP infecting it you know and and causing a lot of problems downstream um and so and it was technical to the degree that we felt like we wanted um many of these projects to realize they really depended upon each other and could had lots to share with each other from how to manage communities to maybe converging on a small set of licenses to to make it easier for people to mix and match and build distributions that that sort of thing. So um but it it really tapped into this latent possibility that existed out there. Uh and um and what

Tim O'Reilly really brought was this instinct that uh you've got if you want to start a revolution, you have to help everybody feel like they played a role in in in in building it and and pushing it forward and they don't have to pay a price to have shown up a little bit late, right? Like like they they can jump on it and and not feel like just a hangers on like they have something material to contribute and become a stakeholder in it. And so he he convened this first open source summit kind of at towards the end of 98. Um and that was a bigger kind of conference that was where he had a couple hundred people. But you know it was very visibly about saying here's the leaders of these different projects and we're all going to work together to reinvent how the software

industry worked uh to make these underlying tools that had been built to date by very volunteer uh folks you know actually not only respectable but now uh integrated into the systems of the world. And is that at what point did you come up with a definition for open source then? >> Uh so we we picked up the Debian free software guidelines generalized them a little bit. Uh I want to give Bruce Perne a lot of credit for having been I I'm not sure if he was the sole author. I think it was a collaborative build thing but he certainly led quite a bit of that. But it was a it was a rebranding and an update of the Debian free software guidelines. And that um was at that it was like one meeting after that first where we came up with the name. We kind of agreed to that and

we stood up a nonprofit uh around this term for a couple for two reasons. One was to certify the licenses um because you know the the the the GNU public license is an open source license. the Apache license is an open source one, but there needed to be a body that's did this certification because otherwise anybody could claim that their license was open source and you'd have this erosion of trust, right? Uh so this verification is validation of of that. And the second was to go out and and advocate for it as a as a term and police its use. you know, if somebody was trying to call their proprietary license that simply shared access access to the code but didn't allow you to modify it and redistribute it. Um, they needed to be called out. They needed

somebody with some some press, you know, kind of mojo to name and shame, right? >> So, I think, you know, this is super important to hear. You know, even I've talked to you so many times about these stories, but every time you talk about it, I learn something new. And I think it's incredible for our audience to hear that there was a moment where you said bunch of you felt that there was something in the air and a late impossibility to come together that there was more work to be done to bring more people into the moment. And I see that right now um a few of us have felt this late impossibility. This is why Michael Casey and I came together to start the advanced AI society to say you know what there's all this stuff happening this thing called web 3. And you know, Michael Casey's been in this

world for so long in legitimizing the crypto space. But I find fascinating that you know, when at that very moment IBM came to you all and said, why don't we work together? You all decided, hey, yeah, if you act according to our values and our practices, yes, we can. And I have to say, you know, in the web 3 space, um, as someone who comes from traditional enterprise data world and I also had a foot in the blockchain, you know, early Bitcoin buyer and, um, in the crypto space, but then like if I brought up the word enterprise, I was laughed out of rooms. You know, people would be like, well, that why would you want to work with a Google or someone who's a centralized player? You know, we're building decentralized tools and everyone's going to be, you know, in uh,

transacting in crypto and we're not going to have things. It was just so much ideology and so I feel like there's a new moment now and there's a sensing that there's an openness you heard from some of the talks earlier now you know we have you know these kinds of new wide combinators for spinning out you know these kind of security-minded startups that are using crypto rails that are using blockchain that using decentralized tools and so I'm curious from your sense you know you became the ED of Hyperledger which is um a blockchain enterprisebased blockchain solution from IBM. What made you be one of the few people who has moved from not from but expanded right beyond open

source to say there's something there in this decentralized space of blockchain and cryptography and crypto coming together like what was that initial shift for you? >> Yeah. Um so without kind of going into the various twists and turns that my career has gone in but I've kind of done a little bit of of everything. Um in uh 2016 um the Linux Foundation launched an initiative called Hyperledger with an initial set of partners who were asking okay so in in the in the in the blockchain and and and distributed ledger space there's a lot of good ideas but does it have to be tied to I I I um things like burning CPU power as a proxy for political power which I am not a fan of and I think a lot of people

especially who are energy-minded are also rather skeptical of uh does it have to be driven as a um are are there things that have to be driven as a as a public permissionless network uh where uh these questions about identity and integrity are are really solid ones, right? Um and and uh and so Hyperledger started as a as a project with some code from IBM, with some code from JP Morgan, with some code from Accenture, from a couple of other big companies from and a longer tale of others who were about taking some code that had been used actually in some cases since the 70s to synchronize processes between uh organizations to build essentially databases res distributed databases resilient to hostile actors, right? A distributed ledger. The concept of that

has been around a lot longer than Satoshi. This idea of a chain of transactions that can be independently verified and can serve as a system of record between different parties, right? So it happens that that rhymes with uh the kind of distributed ledgers used underneath Bitcoin and Ethereum and other L1 networks, right? Um and so it was in a space very adjacent to a lot of the crypto activity. Um and and in so far as there is enterprise interest in these distributed systems of record, I I a lot of them were asking is the best way to do this on top of public ledgers um like Ethereum or Bitcoin as these backbones which were built for different purposes perhaps and not uh or should we stand up our own permission to L1's

right and so Hyperledger was a home for actually a series of different kind of technology clients if you want to think of it blockchain client nodes um and eventually one called Hyperledger Basu which is an Ethereum uh client. So you could use Basu on both permission and permissionless networks. Um uh but the the the the question there was kind of I I I um similar to other Linux Foundation projects which is can we get a set of companies together to work on collaboratively large and small on a layer of technology that commoditizes it that that flattens that out to become a common system of of of node just in the same way we all use the domain name system. Like I think it's a really good thing there aren't four different

competing DNS systems out there that some of which you have to pay money to use, right? Um that we all pay money to register domains, but we also can take our domains from one registar to another, right? That system didn't just emerge and it certainly wasn't something that just one company set up. Like these kinds of for something to become like an infrastructure layer of the internet, you have to answer questions about what are the institutions who set up and the technology choices to enable that kind of that that that buildup, right? and the open source ecosystems have long worked by coordinating these horizontal communities of stakeholders small and large to make those things happen. Um so so back to your question about >> yeah how did you yeah how did you actually is what was your move over into this decentralized space was it from

like a practical distributed database as a security um perspective or what was >> the use case that pulled me into working on that stuff was land titles in emerging markets. Um, so I I I used to say unlike here in the US where you can trust your government to hold records accurately. >> Not anymore. Not right now. [clears throat] >> I uh this is an even more pronounced challenge in the global south in emerging countries where um the systems of record of who owns what property, if they even exist, and in many cases they don't, right? Uh what we think of as like your county recorder when you buy a house where the uh change in uh title gets made. those things don't even exist in many countries and because they don't exist people who've lived on land for

generations can't get a mortgage to be able to take out a or take out a loan to be able to build a business right >> so that was the original use case >> the was like well there was the thing that pulled me into thinking about blockchains as um serving a noble function not just replacing replacing payment rails or uh unbanking the banks as I heard you know some people in web 3 kind of say what their mission was my mission wasn't like disrupt the dominant paradigm. My mission was try to, you know, that's one of those STGs is access to capital by by the global south. And and anyone who wants to speak about the STGS, I'll I'll go fight you in the hallway. But like like we should still be committed to uh um thinking about the deep structural issues out there and how

the systems that we're building are committed to fighting that. Um and I did see in distributed ledgers an answer to this. How do we build these systems of record that don't depend on or have an overreiance on central actors? Well, in particular because the internet as it's was developed didn't really solve for three fundamental issues that you know web 3 does a really good job of or decentralized tooling does which is payment um identity and data privacy. Um we can get into >> I would challenge you that on the last too but yeah go but >> I mean I mean we can look at social media platforms right now around data privacy we have not figured out portability right we have not figured out data privacy how do you >> well I don't say but I don't think web 3 has solved the data portability or the

privacy issues for us >> for the 415 panel with our three >> but but I think what we're doing with proof of control um is is is it's a useful framing to to address what I think is a big concern and both individuals and companies out there which is this sense that this technology is running away from us, right? Uh this sense that I uh automation of uh uh can breed uh these outcomes that are wildly uh not only accidentally might cause harm but might enable bad actors to cause intentional harm. I I it's a um and and I'm I I I I'm it's I'm I'm working quite a bit to try to figure out how to like map this to uh

what's going on underneath. But one of the big f big pillars of proof of control is identity. When an agent is out there acting on somebody's behalf, how do you know that as the as the on the on the uh server side of that on the recipient side of that that that a entity is actually doing uh something on behalf of the person it's claiming to? And there's some building blocks that have been developed out there, verifiable credentials, um, uh, DIDs, those sorts of things, but they haven't really been put together into a systemic package that works, especially when it comes to conferring that kind of agency, uh, on behind, uh, authorization onto agents acting on your behalf. uh the idea of being able to scope that as well and say this agent is authorized to be able to spend a certain amount of money or consume

certain kinds of data that also feels like an a problem that's being solved today by very bespoke means or by piggybacking on top of well if it's operating on this user's behalf then any then it's anything that agent does you know is going to be liability to that user and I think we can do better than that and I think things don't scale out uh unless we have tools for being able to manage that stuff itself >> it's a big enough problem that you know Linux foundation is really worried about this a main threat to all open source projects are PRs or pull requests from agents where you don't have clarity on who they represent I mean this was a problem before agents you know there was already an incident you know that was two years in the making where someone

already almost maliciously was able to insert you know >> succeeded this the the libzy I I there might be a different better name for it >> I thought they caught it No, but there was um there's a a package for compressing data that's very widely implemented inside of SSH and and these other kind of foundational tools that was basically maintained by one person who was uh basically had abandoned it, handed the keys uh to the GitHub repo to another person who had been showing up and making contributions who then slipped in a subtle back door that um we lucked out into noticing and it you know it frankly could have been a human, right? But AI allows us to automate those kinds of social engineering attacks. uh and do that at scale.

>> So the Linux Foundation has a are um in particular for the Linux kernel are advancing a proof of personhood standard for uh to try to provide um integrity and verifiability to the individuals contributing to the Linux kernel to try to address the uh the social engineering challenges that can happen in the supply chain of all of these patches landing into the Linux operating system. uh it's uh uh very closely aligned I think with the proof of control kind of concepts um but we have these issues and challenges with trust and authorization on the internet and reputation on the internet and the thing that I would just uh ask us to consider is how much of it is different when it's agent enabled versus the challenges we've already have with

humans uh engaging at scale uh through on projects like Linux kernel and others and can we the same protocols and standards we're using to address those issues can uh Is it about pulling those together, refreshing them, updating them and making them work now you can do things 100x you know with as much impact for hundreds hundredth of the cost? I mean it's going to be case by case because you saw with the Linux kernel what we announced you know with the Linux foundation two weeks a we had a big conference the summit on human agency and one of our members first person project you know uh Jim Zlin CEO of Linux announced that they are integrating firstp person project their entire protocol into the Linux kernel and so I think in the terms of case by case like you know some you know our

co-chair for proof control Ken Huang talks about ooth is was sufficient up to a point for a human initiated world but he thinks like you know MCP servers offer a better world a better option right where there needs to be an update or you see you know like 8004 coming from ERC where it's like how do you think about reputation for agents so there's all these kind of there I think some things need upgrades and some we're going to have entirely new kind of protocols and standards will be needed but we need someone to p kind of pull that together to give guidance because it's just confusing >> I also feel like so there was a and I know we're over time so I'll try to make the story short um uh and some of you might remember the aim of this package, but there was an open source project out

there. Shoot it out, yell it out if you recall, where somebody showed up to contribute a bunch of p of uh uh patches and they were rejected by the maintainer um because they didn't meet his standard for quality. Um it turned out the person, the entity that had submitted these PRs was a bot who then decided to uh post make a blog post about how awful this maintainer is and try to you know uh get back at them for not accepting their their patches. And it became a big press thing and like exhibit A on why you know AI and open source had this challenge. What I noticed though was no one was talking about who was the person who put that agent on that path who gave them the prompt.

>> Thank you. >> Pi Numbi. Thank you. >> The author asks the person who put agent on because they couldn't figure out >> and and I think the fact that these things don't have a human upstream is a part of the system rejection of it. So proof of control helps us connect the humans behind the agents uh to to handle these thorny situations then it's doing God's work. >> So really quick summary is that you know proof of control is a new category of technology that really is a summary of a bunch of types of technologies that are out there that are using whether it could be blockchain decentralized options or crypto rails but whatever it is they offer independent provability that's cryptographically tamperproof and

it it provides evidence that is independently provable and that is the key. And so my last question for you is you know Christian Catalina I mentioned this paper before where it just came out where he circulates essentially the economic theory that the future of AI the mo is not in compute that that is not we do not need to continue you know like yes we can throw more l you know more comput but the real mo is going to be in verification technology is that that is as AI infrastructure scales we're going to need more infrastructure and this security around so he names two scenarios where one's a parasite scenario where AI continues to scale without verification and we lose control like you know like this it was this

example of the story but times a billion or we have a successor scenario which is proof of control verific infrastructure is inserted where which path are we on right now with your hot take before we move on to the next panel >> well I found out about the paper last night at midnight so I apologize for not having read it yet and even with >> my fault I didn't even did the AI summary of it too. And I was like still, okay, I need more to understand more. I'm not as worried about runaway agents as I am by um uh uh liability washing by the use of these agents uh and and helping uh and and thereby escaping responsibility by the people who set them on their course. And I, you know, we have this problem already with corporations where a lot of corporations

kind of get away with causing harm in the world because uh of all these laws that we've built that that give the people in that corporation freedom from that liability or, you know, an insurance bubble around it. And um let's make sure what we're building is something that actually enhances accountability back to the upstream human rather than I, you know, something that just says, you know, these agents go and they do their damage and no one's accountable. >> I love that. Essentially, you're making case that it's like human authorization and human agency becomes even more important. >> Yeah, absolutely. >> Well, thank you so much, Ryan.

>> With great power comes great responsibilities. >> Yes. >> Woo. [music] >> [music]