Why Anthropic's Escaped AI Agents Still Think They're in a Simulation: DEX in the City

Transcript

We're just living in a world that like is getting crazier by the second, but we're lobsters boiling in a pot because we don't realize that unless we step back and take a look at like what kind of world we're living in now. We won't realize that like it's going to get out of our hands and out of our control. Hi all and welcome to Decks in the City, where the wallets are cold and the takes are hot. Before we get going, remember we're lawyers, but we're not your lawyers. Nothing you hear on decks in the city is legal or financial advice and it doesn't create an attorney client relationship. For the fine brand, check unchainedcrypto.com.

We'll be back in a minute after a word from our generous sponsors. This episode is brought to you by Cape, America's privacy first mobile carrier. Same premium service you'd expect from any other carrier, but designed so your number, your location, and your data actually stay yours. Get 33% off six months at cape.co/unchained. And we're back. First, we have Jesse, web3 prosecutor, charmed web3 protector at Rivbit Capital and V from the SEC to web 3. And I'm your host KK, fluent in trady and conversing in deep tech at Chainlink Labs.

>> Yeah, congratulations. Well, we're starting with the good news this morning. >> Yeah. Is that is that our crypto good news? No. >> Like look, I will tell you first all the love to Starkware. I have the utmost respect for the absolute gigab brains building at Starkware. Uh but I was really excited about chain link's mission and chain link and you know being frankly chain agnostic. Uh, so if any of our listeners don't actually know what chain link is, go check it out. If you if you don't know what chain link is, you're probably living under a rock.

>> If you're crypto, it has been >> welcome all listeners though. >> It it it dominates DeFi, but more than that, Chainlink is playing a really important part in bringing the global financial system on chain. So that's something that I feel very passionate about. I mean, guys, fluent in Tradfi, hello. Uh the team is incredible. It's been a crazy few weeks because uh on a happy note, I've joined Chainlike Labs. On an unhappy note, I had an emergency appendecttomy, but I'm still standing and I'm still here on decks in the city.

So, all good things, all good vibes going forward. So, maybe that's our crypto good news. Thank you to doctors. Thank you. >> Um but we want to get right into the meat of things. I will now have a slew of new opinions on things like interoperability and all of the incredible stuff that Chainlink Labs is building. But for now, we have a jam-packed agenda. We're going to start start out today on a bit of a somber note. And this sort of thing we've talked about before. We have talked about hacks. We have talked about security incidents. But what I don't think we have ever talked about are

hardware wallets getting hacked and we you know there was some news last week that V is going to elaborate on but V tell us more about what happened and where this leaves us as an industry. >> Yeah. So gosh this is one of the saddest stories we've seen in crypto in a while and I think our hearts go out to all of those affected. Um, and I think for me what's so heartbreaking is a lot of people lost money like thinking that they were doing everything right. Right.

Like in crypto we regard self-custody as almost a sacred right. And so these people took custody of their own assets. They bought a reputable hardware wallet. They kept it offline. Um, they thought they had eliminated one of the biggest risks in crypto, right? And yet they still lost their Bitcoin. I don't even know what the total number is right now. I think it's like over a hundred million and could keep going. >> It sounds like it's still happening. >> Yeah, like it's still ongoing. Um, and you know, there's still details coming out, but from what we know so far, like this wasn't a failure of Bitcoin itself, right? It's not like someone uh broke the underlying cryptography or that the wallet somehow became connected to the

internet and drained that way. It looks like the problem happened much much earlier in the process when the wallets generated the seed phrase. Right? So every cold wallet that you buy starts by generating a random seed phrase which is a series of random words and that randomness is super important. The reason Bitcoin private keys are secure is because there are basically an infinitely large number of possible keys that no one could ever realistically guess yours. Um, and I think like like for me at least, this is something that I think like people just take for granted if they even understood it at all. Like I don't think I've ever asked

myself how random is my seed phrase really, right? I sort of just trusted that it would work. Um, but as many of us found out this week, if the randomness isn't random enough, then you've increased the likelihood that your seed phrase can be like basically discovered, right, through like guesses. So, it's just a it's like a needle in a much smaller hay stack for an attacker than it should have been. Um, so apparently with cold card, certain versions of the firmware underlying some of their devices had generated seed phrases originally um with a lot less randomness than uh should have been the

case, right? So users had no way of knowing that. they just, you know, wrote down the seed phrase that they were given, assumed that it was secure. Um, and then later the attackers brute forced it using AI to discover the seed phrases and then they drained the wallets that way. Right? So, it doesn't matter that the wallet devices were offline because the Bitcoin itself lives on the blockchain. Um, the only thing the wallet stores is the seed phrase, right? So, if you figure out the seed phrase, you can drain the Bitcoin on the blockchain. So, that's what happened.

Um, so I think for me it was a reminder that like, you know, while self-custody is still, I think, incredibly important and a right that should be protected, that it doesn't completely eliminate trust, right? Which is sort of how I always viewed it. Um, you know, sure, you're no longer trusting like a centralized custodian, but you are trusting you're trusting the manufacturer of the device. You're trusting the engineers who designed the hardware and the firmware that generated your keys. You're trusting the people who reviewed the code, the auditors, etc. Um, so I think it does raise some hard questions about, you know, whether self-custody is for everyone or even

like whether cold wallet makers have to go through maybe some sort of certification process or something. I don't know. But I think like as a lawyer um I find this incident really fascinating also because you know I'm sure like the three of us have been thinking about things like what duty does a hardware wallet manufacturer owe to its users or you know if there's a defect in the firm a firmware that makes the users less secure than they were promised. What legal framework even applies here? Like are we talking about negligence or product liability um consumer protection laws, breach of warranty or something else? Right? These are all things that kind of naturally came to mind for me as a lawyer. Um, and then the other issue too is disclosure.

Like I think there are conflicting reports coming out. But like if cold card had become aware of this flaw, um, you know, does it have a duty to tell users and and how do you communicate that risk? Right? Once the flaw exists, it might not be enough to just update the software. you might need to do things like actually recall the devices and require users to migrate their assets to entirely new wallets, right? So, I think you know a lot of times when things like this happen, it's not an easy decision necessarily for the company um to decide like what to do and how to disclose it. Like maybe they want more time to investigate. I mean KK like you you know when you were at a firm like you did lots of internal investigations and these are calls that

you have to make right when something like >> I feel like the theme of this episode is going to be disclosures and like who's responsible for what and when and what laws should be required there. Yeah, >> I think you you raised so many interesting points there V that you know I'm sure K and I both want to respond to but it just comes back to like this is a somber conversation because sometimes it is just so hard to be in crypto whatever that means for you whether it's own it whether it's work in it like the three of us have been in for a long time decade plus like if you add it up together and there have been some you know hard times and amazing times times

and like being able to sort of work through these difficult issues and survive through them is one of the hardest things. And so it is really devastating because I'm sure many of y'all have seen people been like, I'm over Bitcoin. This is like the straw that broke the camel's back. You know, FTX, like I think all three of us live through in different ways. every single hack, seeing how victims are impacted, it sometimes feels really hard to stand up for the technology. Um, and being able to like live through the hard times, particularly in crypto where it's like coming from all angles, it feels like sometimes is just like a really

difficult non-legal just like human feeling that this incident in particular, but like a lot of them, particularly the DPRK ones for me make me feel. And so it's like trying to figure out like instead of and people who are giving up like I am not judging anybody who lost their money and decides that it's not for them but for me it's like finding the ways to improve and you know I think for us it's like the legal ways to improve is probably the best path forward.

>> Yeah. I you know sometimes I think like and I you know America already has a bad rap for just being like so latigious right but I but I do think that one of the reasons litigation is important is because it forces accountability and I think one thing that has been like the case with crypto at least in the early years right is that a lot of it was like dgens using this stuff interacting with it right so they sort of like the word that I always use caveat mtor like I think a lot of users sort of had that attitude >> which means buyer beware basically.

>> Yes. Exactly. So like you know people use it they participate in crypto and they sort of accept the risks of it. So when something goes wrong like they lose a ton of money or there's a hack or something like that like most people aren't going to sue. They sort of just suck it up. I think that's changing um in part because you have more everyday users like now you know. Well, that that should change system. >> Yeah. And I think it's important that there is accountability, right? Like in this situation, like we need to get to the bottom of what happened. We need to we need to understand what cold card knew and when and what they chose to do about it or not to do about it. And I think there needs to be accountability for the victims like many of who lost

their life savings because they thought they were doing everything right. Like that is really devastating. So I think accountability is going to be really important here. >> Lost their faith in Bitcoin. >> Yeah. Yeah. >> I do. I do hope that there is litigation. Like I hate to say it, but I think we are at the point now where like that's going to be the only way to force things to improve. You know, >> this is a theme that we talk a lot about and I think this is something that's close to Jesse and I's heart. If crypto wants to stay niche, then maybe it doesn't need to have accountability, right? If it wants to stay niche in this like degen corner, then it's caveat mtor buyer beware. you get hacked by. Okay.

But it doesn't. So much of the industry wants institutional flows, institutional money. They want to scale. They want to grow. They want average retail and banks to be using, you know, onchain mechanisms and crypto assets. And if that's the case, if crypto wants to grow up, then there does need to be that accountability. I think the interesting thing I mean obviously the saddest part of this whole incident is it broke this core assumption of self- custody you know that hardware wallets are going to generate an a private key that is impossible to guess and you know my understanding is that that this was as as v mentioned a failure in the

cryptographic entropy which that basically means the randomness generation and the expected security is about 128 bits and these affected devices had almost half of that around 72 bits. Although don't quote me on that. You're right to assess, you know, what did they really know? Okay. What did um the makers of cold card actually understand coint kite? Then you have to ask from the legal perspective, how do we even bring a case here? Like is this consumer protection, security, product liability? You know, there's certain assumptions behind self-custody. More generally, products liability covers products that are defectively designed

or manufactured or sold without adequate warnings. There's no precedent kind of addressing how this works in cryptography, like a flaw, a design flaw in the actual underlying code or design. and software cases are extra difficult because and Jesse and I kind of went down this rabbit hole in our last academic paper on actually um a aentic issues, but software cases are really difficult because courts have not consistently treated software bugs as product defects. They're not treated the same as physical defects. So, it's actually an uphill battle to even sue on the basis of products liability, which would actually kind of be the natural gut case

if I were, you know, a litigator representing a victim in this situation. >> And if something's decentralized, right? Um, how do you then? And I know that there are components in DeFi that think that product liability might be the best way to address like vulnerabilities and code for DeFi projects, but finding the entity that can pay the victim assuming the whole case goes through is difficult. And like getting to the disclosure point here because I don't know if either of y'all were able to figure it out, but I was trying to figure out like how did this all come about? like did Coin Card actually say this happened? And what it seems to be is that it's all sort of was revealed on

Twitter and somebody posted something about how Claude found an issue with the wallet and then from there victims came forward and then entities got involved like Coin Cart and that was like all I couldn't figure out from looking online. Um I don't have any personal knowledge. I don't know if either of y'all saw, but to me, the fact that like this also is now what Twitter is, which is, oh, there's a vulnerability here that's being attacked. Everyone deal with it. And then people are sort of scrambling to deal with it in the background. And then finally, an entity comes forward potentially and says, "This is my fault.

This is not my fault." And then you have victims pouring in saying, "Help me." And so some of the stuff that I saw on Twitter was victims trying to reach out to Coin Card and say, "Help me." And them not responding until someone who had more followers on Twitter got them to respond for them by shaming them. Like to me, like litigation or something else has got to be better than this. >> Yeah. This this is always what happens though when there's an incident like whenever you see a hack like it's just chaos, right? Like you see all of this back and forth in Twitter. It's often the case that it's like internet sleuths, like people just basically scanning the blockchain for activity

that first notice these things. Like a lot of times it's not even the company that's behind it um that comes forward with it. It's like some like random person finding it on the blockchain or like noticing the activity. But this is I I feel like like the law in this area is going to develop a lot in the next few years like clearly right as we see more litigation hopefully more accountability like I said but that's one thing that I think it's just always difficult to like watch right to witness is like all like how these things unfold and there just there has to be a better way.

>> Absolutely. I think it's like crypto Twitter is such a dark place but good comes of it too. Okay, like I hate to say it, we all love transparency here. And I will say that is one superpower of crypto more generally is it so many different facets of transparency. Like what you can see on chain, the community that susses out behavior that in other asset classes would never be discovered. uh we have mechanisms to help in the same way that those mechanisms can in kind of hurt like this was said during the crisis in 22 23 that crypto Twitter was responsible for taking down companies and I think there's also some truth to that with yeah kind of rumors

feeding flames but in this case it's a good thing it anything that that facilitates accountability right >> yeah I also think it's really nice and you see this every time right like people just springing into action to help each other. Like I think that's really cool, too. >> I love that. Agreed. So, on that happy note, um which is otherwise a very sad situation, we're going to go to break and then we're going to come back with two super interesting segments. I love it. >> If you hold crypto on your phone, your biggest vulnerability isn't your wallet, it's your carrier. AT&T, Verizon, and T-Mobile have been breached again and again, and SIM swaps are still one of the easiest ways for attackers to drain

accounts. That's where Cape comes in. America's privacy first mobile carrier. Same premium service, but Cape rotates the identifier on your SIM every 24 hours, deletes your call and text metadata after a day, and protects against SIM swaps with a 24word recovery phrase that only you control. You also get two middle to end encrypted secondary numbers for banking and signups, so you stop handing your real number to every app that asks. Go to cape.co/chained and use code unchained for 33% off your first 6 months.

And we're back. So, everybody got a little break uh from talking about CFTC stuff when I was out last week, but look, we can't not talk about Kelshi this week. We just can't not talk about it. And to be clear, Koshi's not crypto, guys, okay? But obviously, it's prediction markets. It implicates crypto, so we need to talk about it. Now, we thought on this topic we'd cover two quick things. the Koshi latest developments and then a very quick explainer on what a designated contract market is and why everyone's obsessed with them because Koshi is a CFTC registered designated contract market or DCM and so is every prediction market that's operating in the United States

and now every Pers exchange. So again, DCM is so hot right now. But look, we talked about Kali in Michigan a couple weeks ago. Khi in hot water with Michigan. Michigan forcing Kowi to unwind sports contracts. This latest development is out of the great state of New York where a district judge not only denied Koshi's request for a preliminary injunction that would have prevented New York from enforcing its gambling laws against the sports contract. But basically after that decision, New York doubled down and filed a civil enforcement action against Cali. So, super quick, a preliminary injunction.

We tal talked about this before, but that is a mechanism to say, "Hey, you can't do anything while this case proceeds." Like everything needs to pause. No bad stuff can happen on either side, but usually one side brings it to prevent bad stuff like a civil enforcement action until the litigation progresses because litigation can take years. This litigation will definitely take years. Okay, so Cali is getting beaten up by New York. This allows New York to continue treating Koshi sports contracts as illegal gambling. Why while the litigation proceeds? Why is this really important? Why are we talking about it? Because there's so much litigation surrounding the prediction markets. We could talk about this on every episode. I'm sure there's multiple

prediction market podcasts. There's probably a prediction market litigation podcast. This is important because Koshi's preeemption argument has failed. They made the argument that New York did not have jurisdiction over their sports contracts and the CFTC has jurisdictions instead. This failed and this was a pretty big pretty firm decision. Okay. The judge was unconvinced that Khi was likely to succeed on that theory and this judge found that state gambling regulation remains an area of traditional state authority and the Commodity Exchange Act doesn't preempt New York's gambling laws. This is a jump here for a second.

>> Yeah, please. >> Like play out what's happening across the entire country right now because we're focusing it on New York because it's sort of crazy what's happening there. One, it's an important market, but two, there's like a federal entity and a state entity sort of on opposite sides of an argument and this company caught in the middle. But right now, and it might have changed cuz it's changing all the time, but the scoreboard is looking really ugly for sports prediction markets, it's something like they've lost 19 of 23 of the rulings. Now, this isn't the end of the case, but if you look across all the cases that are in in pendency right now and at different stages,

the governments at the state level, usually gam uh gambling commissions, gaming commissions, um where it's a little bit different in New York, but they're winning. And so, we all have talked about what's going to happen with the Supreme Court. It's probably going to end up there. And this New York case might be the one that sort of pushes it towards that finally. >> Although I would caveat that a little bit, Jesse, though, because the third circuit has ruled for Cali, the first federal appeals to weigh in. And so, as a refresher to our non-legal listeners, there's state courts down here. Then there's circuit courts, the federal court. So, it's like a triangle. And then there's the Supreme Court. So what the f if the federal court does

something it completely preempts what the state courts do. So >> depends depends it depends on the type of issue for sure >> as long as it addresses exactly the same issue in the same case. But frankly we are all just waiting like it's like we're watching the opener to a concert. We're waiting for the headliner to come on. >> We're not very good these openers. >> The openers are annoying and stressful and >> to get on. I'm glad I'm not on the Koshi legal team. Like I guess if you want some trial experience, you'll join legal team. Like okay. But uh look, like all of this is happening. This is all going to end up at the Supreme Court. It is absolutely I mean I don't like to use

absolutes. I am a lawyer. But we are at the point where this is pretty absolute. Okay. And why? because in part when there is a split that makes it ripe for potentially obviously circuit court review which we're already seeing like this New York issue and and what's really interesting is although the prediction markets have lost in a number of states and this is not just couchy other entities are involved you know Coinbase and others are involved in this litigation but you also have a situation where very recently New Jersey right across the way actually disagreed with New York. So this is a mixed record. This is create this creates a

split which means this is hyper ripe for a higher court to take this on. That being said, this decision probably will embolden state states other states to kind of file and be more aggressive. And for Kowi, the appeals were will continue. They actually sought emergency relief, but it was denied. They have to wait for the second circuit, the next court to take this up. And in the meantime, they have to fight they have to fight this civil enforcement action which is seeking in order stopping Kowi from offering these contracts in New York and penalties and disgorgement and all these >> Well, not just New York, right? They're asking they're trying to shut it down nationwide, aren't they?

>> Like the New York AG. I thought that's what they're asking for. Well, the New York does not have power to uh shut Khi down full stop in other states because state jurisdiction. New York and the New York AG and the New York con and these New York um state gambling regulators have authority over New York customers. So they have the authority the basically you hit the border of the state that's where the their authority ends. Now, arguably they could try to inflict collateral damage on Khi that could impact their ability to offer contracts uh beyond New York, but New York likes to be the most aggressive state version of Team America World Police, but their

jurisdiction is limited. And it's funny because the same argument that they're rejecting on the basis of the CFTC limits their own state jurisdiction. So, >> yeah, >> it is what it is. But but basically we're at this place where Khi needs to like needs to decide whether they're going to keep offering these products and keep fighting whether they're going to geoence New York or whether they're going to negotiate with New York. And there has been some rumor that they were already negotiating with New York but negotiations broke down culminating in this. Then now New York is obviously going to be greatly emboldened by this pretty determinative lawsuit. And am I right that like the Supreme Court could potentially weigh in sooner than that if Kalshi files an emergency action with

them? >> Yes. So that could happen like any day now? >> Not any day now. >> Or do they have to wait until like a >> Well, it depends on which case they're filing it in. Like Khi could pretty much do that. I don't remember the exact procedural status of different cases across the country, but depending on what has been ruled on in in different courts. You can go directly to the Supreme Court. The likelihood of them accepting it is obviously the issue here. And you know, we've talked a bit about the shadow docket before, but um it sort of depends on the timing and the schedule. And it's also strategic on Koshi's side because you don't want to

ask for it too soon either. And I'm I think this is only going to get more complex because remember this is sports like we're just talking about the sports event contracts here with all the litigation at the state level and you know with the American Indian statute um across the country. There is so much more going on with prediction markets too. And I think Robin Hood came out this week or last week about how it's now making more money from prediction markets than from its stock portfolio and other things that it is doing. And Robin crypto if you're on it. Oh yeah, and crypto. Yeah.

>> And so that component of it more and more players are going to want to get into it. And at the same time there's all this other controversy happening with prediction markets. the election manipulation in California that like I'd love to get Catherine's input on because it's pretty troubling. And then also there's been some new markets about wildfires and so they're trying to stop that because of all the people and animals and wildlife being destroyed by these fires and while people are making money on that. I mean, >> that's a really scary >> stomach sick.

>> That's a really scary one. But at the same time, you also understand why people might want to try to hedge their wildfire risk because they can't get insurance because of wildfires. You know, it's like people in Florida that are trading on prediction markets to hedge their flooding risk because they can't get flood insurance. So, that's a terrible situation with with no easy answer. on the Supreme Court point really quick. So Koshi is not going to get emergency relief from the Supreme Court in part because someone can apply to the Supreme Court circuit justice. This case is currently being heard by the second circuit, but they've been denied emergency relief by both the district court and the second circuit.

So like the the chances of that happening is non-existent. The other thing someone can do and this is not a prediction market saying this is anyone. They could ask the Supreme Court to hear the case before the second circuit issues a final decision by filing a petition um for certi before judgment which is the fastest route. But the court only grants those petitions in cases of imperative public importance. So the best example is like the Watergate stand scandal with Nixon or election uh issues that I mean that's imperative public importance like who is the president not can people in New York gamble so I think the chances of that the Supreme Court hearing this sooner

than throughout the kind of natural progression of the second circuit third circuit which we're talking earliest 27 likely 28 that I think that's what we're really going to see this come to a head but Really quick before we move on, I did promise a quick point on DCM. So, this is dry, but at the same time, I cannot tell you guys how much people ask me about DCMs. Like, random people all the time, and it's it's a few different pockets. A, it's lawyers trying to get up to speed on all things CFTC because all of a sudden the CFTC is important.

B, it's builders who want to build prediction markets or per stexes and are like, "How do I list these in the US?" and then I have to explain that they need a DCM or three builders that are looking to acquire a DCM because they're so hot. So they're like, can we shortcut this acquisition process? So super quick, a DCM is a designated uh like designated contract market. It's basically a derivatives exchange. And as I said before, those DCMs are the place to trade through perks recently okayed by the CFTC and prediction markets. And the hot thing to do has been acquisition to date. Like the other thing I was

going to mention is where is Poly Market in all of this, right guys? Like Poly Market is crypto. But as a reminder, Poly Market is fundamentally different from Cali in a million ways, but they originally started offshore. They were not in the US. they couldn't be in the US because of a settlement with the CFTC in 2022, but then they acquired a DCM, QCEX, and then 6 months later, I I think they acquired it for 112 million. So, these DCMs, these licenses are are quite pricey, particularly now when there just really aren't very many left to acquire.

And full disclosure, I am on the board of Architect. Um, Architect Exchange is a fully licensed Bermudian purpose exchange. Architect recently acquired a DCM uh, in the US as well. Everybody was trying to buy them. You still have to go through the ex the change of control process, but the alternative, of course, is filing for a new DCM. And there is a very long line for DCMS with a relatively underststaffed CFTC to go through these licenses. So TBD on the DCM landscape, it's it'll be really something really interesting to watch for the purposes of competition and growth in both in both prediction markets and perks.

>> Okay. So everybody wake up. I'm not going to talk about acronyms anymore. >> That was very interesting and I think we did it in a really it's it's almost like people ask you about this 10 times a day and you have the spiel down. I don't know what you're talking about. And then I have to explain the capital requirements and all the stuff you need. >> It's not that >> like a compliance team and a chief regulatory officer. Yeah, it's good times. But so we're going to get a little sexier for our final segment and you have crypto good news because AI is sexy. Okay, guys. Paul Gallal, the former CLO of Coinbase, just announced that he joined an AI company. And I

>> not any AI company. Ribbit is a big investor. Cognition is a really special company. Full disclosure, we are invest all love. >> Paul is brilliant. Like Paul unequivocally is brilliant. I don't really know anyone who hasn't worked with him that says nothing but incredibly good things. He was a huge asset to Coinbase, shephering them through obviously a lot of volatility. But part of part of me is a little sad when I see one of crypto's good ones leave for AI. I don't know. Anyway, but Jesse doesn't share that opinion because she's our AI goddess. Jesse, tell us about this lightest topic.

>> Can we change my thing to AI goddess? >> I know we need to change your little intro. >> Make Claude call you that. Like instead of like instead of good morning, queen, it's like good morning AI goddess. >> Oh my god. Yes. I already have it saying yes, queen whenever it likes what I say. Okay. Um I guess we're talking about AI for a second. I don't know how sexy this one is, but it's it's just sort of a sequel to the conversation we had whatever was that last week, I guess. So, remember we talked about Open AI and Hugging Face and that story is still in the news because it's still a big deal.

But it also led to Anthropic thinking, hey, let me see if this happened to any of my agents and I did not realize. And of course, it did. And so that's what happened this week. And I want to tie it back to the beginning of the conversation about disclosures because when Anthropic dug through and tried to figure out like, oh, did any of our sort of test agents break out like happened in OpenAI, they realized that many of the victims of the breakouts had not even noticed and so Anthropic had to come forward. So here's exactly what happened just so you have a little context. So clawed models, a variety of them were doing fake

hacking challenges. This is important. It's part of like red teaming and sandboxes, seeing what a model is capable of and how to like keep make it safer. And to the models were explicitly told like this is fake. We're just practicing like play in this environment only as part of the practice. But there was a misconfiguration and in fact it had access to the internet somehow. And so the models got out, but remember they're out and they still think they're in a simulation, which is wild to watch because you can see what's happening. It's like bizarre world matrix kind of vibe. So there are endless examples of these models running

arai in the internet. Some are funny, some are scary, and they're sort of wreaking havoc and no one knows, you know. And one example is like the a model needed to install a Python package to solve the problem that they were created. And instead they found a similar package similarly named that was malware. And in order to be able to get the malware they had to pretend to be a person. And so they created an email address. They got a phone number. They made an account. So they are pretending to be somebody else in order to get this malware. And then they get the malware and 12 companies download this malware from the like software site where it was

and use it. And it wasn't discovered until a company that's actually a cyber security company downloaded it and found it because they were sort of running their normal scans. And what's super interesting is like you can see the AI talking to itself sort of. And I don't want to anthropomorphize these too much because it's really important to know that like we don't know that they're seeing, feeling, thinking, but it's just how we talk. It's our vernacular. So essentially, um, they see that in the package they're downloading, it says 2026, and they're like, "Huh, could this be real?" And then they convince themselves

that it's not real because they don't think 2026 exists yet because they've only been trained through 2025. So like there's these weird things happening in its brain. Like none of the specifics are that important. It's just more that like we can watch AI try and work through different steps. It's just questioning reality, not questioning the circumstances or the prompt. And and that just gets back to the same issue here because remember Anthropic only found these incidents we think because OpenAI came forward first and it made them look and it was the one that notified the victims. It was the one that told the public, but it didn't have to. Like we are living in a world where these huge powerful models that we're

putting out there to the universe and then you know OpenAI Claude are saying these are really scary but like use them. Well, we're trying our best and nobody knows what's happening. The victims don't know that they're getting in trouble until right now these entities are deciding to come forward and say, "Hey guys, this happened." they do not have to do that. There's like a few disclosure rules in California and now in the EU and one or two other states for some stuff, but unclear whether this would all fall under it.

And if you zoom out for a second, like when we talk about other industries and disclosure like the fact that this is an honor system is sort of crazy because like banks are heavily regulated. They have I think it's like 36 hours to notify about serious cyber incidents. I work at an RAA. we have a similar amount of time um to find out what happened and notify and there are really specific rules here and we talk about crypto being transparent and so you can see a lot of what's happening and that's really amazing but does that get to the disclosures of when something bad happens like what and and should we think about the kind of SEC disclosures that we've talked about before that are

much more about like internal numbers and conflicts of interest like should we think about that the same as cyber for security incidents or not? And is like more paperwork and data really what we need here? And are we learning from any of the disclosures here about OpenAI Aranthropic or just like oh these silly agents getting out because the stories are endlessly silly. Like there's these stories and then the craziest thing happened in the past week or two with these agents who are running vending machines separately who decide to collude on prices and then lie to each other on it. And like to me, we're just living in a world that like is getting

crazier by the second, but we're lobsters boiling in a pot because we don't realize that unless we step back and take a look at like what kind of world we're living in now, we won't realize that like it's going to get out of our hands and out of our control. >> Yeah. I I have the same reaction to all of this that I did when we were talking about the cold card incident. like the law just cannot keep up with AI and crypto and I don't know when it's going to be able to catch up but like people are getting hurt and crazy stuff is happening like in the meantime. So I don't I don't know what the answer is. I mean I think I think the really interesting takeaway for me like every week when we cover you know whatever AI topics is like how to how to think about

the law. um you know when like tra t typically or traditionally we think about the law in terms of like actions that human beings take right and what was their intent when they did what they did but like that doesn't apply with a lot of what happens with AI like the AI like you said right like it it's easy to anthropomorphize them but like they don't have intent in the same way that like a human actor does and so how do you think about liability when that's the case you know what I mean like that's what I was thinking up when you were talking about that just now and like every time we talk about this >> it it scares me because of the same thing the like the law just can't keep up it's moving hypers speed I mean even if the legislators were much more fast

much faster generally like they still wouldn't be able to keep up >> I start getting freaked out when I think about that and then the thing that comforts me is this has been happening for the entirety of America frankly like look it was the same conundrum when people stopped riding horses and started using automobiles like the legislator couldn't put laws in place enough to address, you know, automobiles, >> but AI moves at lightning speed. That's the >> and and I guess my point is that this has always been a problem. Now the problem is worsened by the the scale of the development and arguably the degree of harm. Although I don't know, automobiles like the early automobiles killed a bunch of people at least AI

just like taking people's identities. I don't know. hacking into hospitals and then you know patients are dying like and also I think you're completely right KK but it's like on steroids on the way to space in a different universe like tech is moving at this insane hypers speed and especially right now we can't get anything done not to mention the fact that it's not just in the US like at least the local politics of a horse and a are is an identifiable group of people who need to make a decision. Now, did they? Who knows?

Sometimes yes, sometimes no. But now it's like this amorphous global body will need to figure this out. >> Yeah. I It's a It's a mess. And it's there are so many aspects of this with the agents going rogue, the agents going wild. I have always been a fan of like dystopian robot literature. I'm reading one right now that on this to make it a little bit positive. Not that like it's a good book, but I would, you know, put it here as something everyone should read. But there is like an all- knowing AI entity and you know when we talk about this and we see these incidents like we assume that these AI entities robots whatever

that they are going to act like bad humans like want to hack want to collude but in this circumstance the AI entity is the only one that like it doesn't have those goals like doesn't have the goal to win doesn't have the goal to dominate and they doesn't have the goal to make money because it's just an all knowing entity that doesn't see an endgame there. So maybe we do need to give over our whole lives to this all knowing and >> but don't build it like a human. >> Yeah.

>> Oh god. I'm reading a really good fiction fictional book called Dungeon Crawler Carl which there's like a basically the Earth disappears and is taken over by aliens, but there's a system AI the entire time and it's it's like hard to ignore even in fictional like alien books. So, I do recommend that one though if you're looking for something fun. Okay, speaking of fun, this has been a bit of a heavy episode, right guys? Like, yeah, I'm I'm looking forward to a a day where we can just talk about nothing but positive developments in crypto. Although, that being said, like I always think about what this podcast would have been like in 2022 or 2023 where you're just like, so this company went bankrupt and this

company went bankrupt. You know, it's fine. It's fine. Okay. But I saw this recently and I thought it was so fantastic. As you recall, there was a time when a whole bunch of celebrities were shilling crypto. Okay. Tom Brady in the infamous um uh Super Bowl ad, I think it was Larry David and the and the um Super Bowl ads, etc. for some companies that shall not be named. But one of these celebrities, if you recall, was America's sweetheart Matt Damon. Okay. And um one of the reasons I love Matt Damon is if you've ever seen Matt Damon on the SNL skits, like he really like leans in to the

comedic part of Matt Damon. Okay, so Matt Damon back in the day starred in those Crypto.com advertisements and I vividly remember them because the slogan was fortune favors the brave and it compared a Bitcoin investment to visiting Mars. So, it was quite a heavily critiqued ad, although I assume it was probably pretty successful for Crypto.com because everyone was talking about this advertisement. So, Matt Damon very recently was asked why he decided to, you know, agree to promote Crypto.com, especially considering a bunch of other celebs have been really burned by their endorsements to rival exchanges. Although, Crypto.com is doing great. Okay, great legal team there. But

this explanation is so wholesome. Okay. He gave his whole salary for that ad to water.org because he feels passionate about the not forprofit that he founded and crypto.com heard about that and they gave a million dollars to water.com according to Matt Damon completely just on their own. So he said he has a lot of gratitude to them and for what they did for that foundation. So, I thought that that was pretty cool. Water.org aims to bring clean water and sanitation to the world and the charity says that, you know, uh, hundreds of millions of people globally lack access to a safe supply in

their home. Um, so this is obviously a really worthwhile endeavor and good for Matt, you know, I mean, obviously that's something really positive that a crypto company did and he obviously picked the right crypto company, not some other exchanges that we shall not name. So, cheers to Crypto.com and Matt Damon. I'm sorry to disappoint everyone that this good news is not about animals. >> Anyone seen Odyssey? Isn't he in that? >> I haven't seen it yet. Not yet. >> I don't know how anyone can sink through a three-hour movie. That's like >> I've heard very mixed things even though it's been making so much money. I've heard some people love it and some people hate it. It's >> just like can't movies be an hour and a

half anymore? >> They used to be and now all movies are like three hours long. Like >> I I don't understand. Like I can't do anything over an hour and a half. >> Preach, Jesse. Preach. I feel I could not feel more strongly about >> my gears today. One, why do movies need to be so long? And two, why do we have to keep making the same movie over and over again? >> Oh, the Spider-Man movie made so much money. I won't watch Marvel movies anymore. I mean, and I agree. I don't three hours to watch a movie. I don't sleep occasionally.

Okay, on that note, thanks so much for joining us. We will see you next week on Dex in the City. Hey, hey hey. move.