Does an Escaped AI Model Prove No Sandbox Is Safe? - Uneasy Money

Transcript

So, the our next story is OpenAI model hacks hugging face, right? So, um I was I was telling my wife about this when we're having dinner. My son, who's six, goes, "That doesn't make any sense. Can you explain that to me in like um you know, explain it like I'm five sort of thing, right?" Um and I was like I was like, "Okay." Here's the equivalent, right? Just to understand how [ __ ] crazy the world that we live in is now. Imagine that Apple is building a new iPad, right?

And they like don't know how good the iPad is, right? Um and so they put it in like a room and lock it in the room, right? And they like give it some things to try and do, and it [ __ ] grows legs, climbs out the window, runs across the street, and like steals a bunch of phones from a mobile phone store instead of like calling people and like [ __ ] like doing pranks on them. That's how crazy the world is that like a piece of technology that we built, you put it in a room, and you can't trust that it isn't going to escape out of there. And he was like, "That's pretty crazy." And I was like, "Yeah, it is it's it's actually even crazier than that. So,

um OpenAI, they're testing some stuff out. They give a model no access to anything, and it's like, "Well, this is annoying. I really need some access to the internet. Let me just have a little poke around here and see if I can I can figure this out." Uh so, I don't know like do you want to give us the details on this, Tay? Cuz this it's [ __ ] wild. Like this is like the AI doomers like wet dream, right? Like >> Yeah. Well, let me Okay, so I want to say we've talked about this before.

You're actually I think the one Maybe it was you who pointed out sometimes they get quite fixated on things and they just like go. >> Oh, yeah. Yeah, yeah. >> So, and I think that that's what happened here and there's a lot of commentary on the timeline right now that we should talk about up front where people are saying that either this is like a marketing thing by OpenAI which is Okay, sure, whatever. If you want to believe that. Um, but the other one is that people seem to think that there's no way an AI would ever take this route and we'll talk about the exact route in a second, but like take this harder route rather than the easy route. And the easy route being just completing the task at hand,

right? So, keep that in mind cuz we're going to come back to that. So, here's what happened. OpenAI is they train models and then they like benchmark them and they fine-tune them and so there's like a whole circle thing going. One of the ways that they sort of like evaluate the in progress models is with especially with cyber capabilities is this thing called Oh, shoot. What's it called? It's Oh, exploit gym. I love it. Exploit gym. Okay, exploit gym basically >> where they can train themselves up to be able to do a better exploit.

So and >> And so what it does is it's just like like most of the benchmarks, right? It is a standardized sort of way to evaluate the models and so you give them a very short and succinct task which by the way when it's exploit gym, the task is to exploit. So, the task is like given this very limited set of information and access get full RCE, meaning like full escape, whatever. Like do the thing from this bit of information that you have.

Uh, and the information sometimes is like like log outputs, crash outputs. Sometimes they're given like right? Cuz they're actually given access like a system. They're not This not like a fully autonomous testing thing. Like they're just supposed to hack the the little this little tiny thing. >> The tiny folder. >> It's Yeah, it's like a capture the flag, right? Okay. So, this is where it gets weird. So, OpenAI has a system dedicated to that, right? And the system, because these models like don't have any controls on them, right? They don't have like the You can't do cyber things.

>> missions, whatever. Yeah, yeah, just like >> Right? We get the nerfed version, but there's a version that exists in like this server sandbox somewhere. That's where they're doing the testing and the benchmarking. And this is, by the way, an in-progress model. So, this is the model doesn't have a name and it's a it's in front of uh Soul 5.6, right? >> Yeah. >> Um although I guess Soul 5.6 was also in that same sandbox, so they were together, which they [laughter] They already knew each other's situation. >> Yeah. >> Right. So, they're they're given a task.

They're told They're One of the tasks, whatever it is, it's like go exploit this this thing, right? Okay. The model realizes that it's being benchmarked, okay? And is like, "Okay, I need to do really, really good." And somehow, at some point, the model decides that instead of just [ __ ] answering the goddamn question and hacking the thing and completing the task and getting a good score, that it would be more fun. Let's go with more fun. >> Yeah. >> To just get the answers themselves. And then they went on a crazy autonomous orchestrated journey to escape their sandbox. Okay? So, that was the first thing is that they found they're allowed

to go grab certain packages. And the package Therefore, it's a sandbox, but like there's egress >> There's They've got There's tools in there, right? Like they're allowed They're not there constructor on hammers. They're given a hammer and some nails. >> So, they're allowed to go talk to a package manager, which like we'll assume it's either NPM or maybe it's a local host at NPM or whatever it is, it's not in the sandbox with them. They have to talk to it. So, the first thing is like, let me get out of a sandbox cuz this thing is annoying as [ __ ] There's no [ __ ] internet in here. And the way they do that is they find Okay, so here's the sandbox. Here's the egress ingress, right? For for the package manager that's over here. They find a zero day in this [ __ ]

And they're like, "Haha, now I'm out of the sandbox." So, now they Now they're out of the sandbox and have internet. The next thing they do is like, "All right, who writes these stupid tests that I'm being told to do?" And they find out that it's hugging face. Hugging face and exploit gym, right? They are the benchmarkers. They are the ones if I go hack exploit gym and get the answers, then I can go back to my sandbox and do really well on this test. >> I bet you can see the logic of it, right? Like, if it solves a problem itself, if it solves the problem, which it it easily could, clearly, it's smart enough, right? It will never know. It won't know, right? It's looking for certainty. It won't know if like if I

said to you like, "I'm super good at math, right? Give me like a 10-part question. I'll give you the answers." You Like if I knew that I was being evaluated and wasn't going to get told how I did, that you're just going to walk away, that would be deeply unsatisfying to me. >> Yeah. >> I'd be like, "No, no, no. I won't do this unless you tell me what the answers are." So, I can see how it could be like, "Fuck this. I'm not playing this game where you trick me." >> It's the most engineer [ __ ] brain thing. I've I've worked with these guys for years. Actually, my dad is basically one of these guys. Like, there's a way to go about things.

Like, in my example is always like, I'll sit there and copy-paste things, right? I'm like, "Ah, if I copy If I just copy-paste all this data of I need, it'll take 5 minutes." But back in the day there would be like five engineers calling me a [ __ ] idiot and then they go spend the next 5 hours writing a script to >> scrape siphon the stuff off. >> Yeah, just scrape the Yeah. And then the script didn't work and then they build a library. >> And like 3 months later they're like, "I've solved the problem." >> "I solved it." And I was like, "Yeah, yeah, I solved it." And I put on I put on a freaking my favorite podcast or my favorite song and it was done by the time the podcast like, you know what I mean?

Um so it's like it's just it's a very engineer brained way of of approaching things. It's just like how they're wired. They're wired differently than the rest of us. Okay so this is where it gets super interesting. Well, I mean it's already [ __ ] the best story I've ever heard in my entire life. But Okay, so he gets out of his sandbox. He figures out that hugging face exploit gym is has the answers. And so the next thing he decides to do is like, "Okay, well, clearly I need to go into the hugging face exploit gym to get the answers." And so they start poking around and they find another zero-day and they exploit that zero-day to get into the hugging face infrastructure. So they have now escaped the sandbox, found a zero-day here to get onto the internet.

Wandered around the internet. Decided that it needs to get in here. Found another zero-day to get into the hugging face servers. And then we know from hugging face's first statement that they put out, this is like the hugging face guys put it out before they knew what was going on, is that they spawned uh an absolutely massive amount of what they call like short-lived containers, sandboxes. And that's what was like executing the attack on hugging face. And the attack being So in crypto, right, you get like the attack is like you can imagine like a thousand transactions to siphon the money off. In hugging face's case, they want the answers, they want the data, they want more access.

The thing that was doing that were not transactions as it would be on the blockchain. They were these little autonomous sub-agents that were spinning up and then like doing a thing and then disappearing to probably like to evade detection and stuff. Yeah. Um Now, one, amazing. I Like I don't know how much like how much of this is true, how much of it is missing, but like I don't care. It doesn't matter. It's true enough in my book. It's freaking great. I It's like I love this story, okay? It gets better though. And concerning, okay? So this is where we get some cute blocking the story for the next couple weeks cuz the government's going to come in and it's going to be bad. Okay.

While the OpenAI 5.6 and let's call it 6.0 >> Yeah, 6.0. >> unnamed future model that's really good >> like reaping havoc. >> Reaping havoc meaning that they've like escaped and they're like running around the internet and they're running around hugging faces systems all on their own. Apparently OpenAI did not notice this for at least some amount of time because specifically because hugging face noticed, detected, shut down, and started to started to investigate, and wrote a blog >> Yeah.

>> before they got in contact with OpenAI. >> I'm surprised the models didn't go and just delete the blog post. >> I mean, how did OpenAI not front run this? Like >> They hacked like uh yeah. >> Oh, so bad. Okay, and in that blog hugging face was like we haven't figured everything out yet, but we we wanted to be up front and disclose. Also, we don't know what the model's being used, but they definitely were not hamstrung by any like cyber safety things. And they're like, but we were. When we were trying to investigate, we literally could not investigate or remediate with AI because every single model was like >> no, sorry. Can't help you.

>> So they spun up a what? A GLM 2 5.2? >> Yeah. >> They He up a GLM 5.2 and ran their investigation through that, which is obviously insanely ironic considering they were up against GPT unnamed 6.0. >> Yeah. Yeah. >> Like that's not, right? If you like this clip, tune in to Uneasy Money every Wednesday at 4:00 p.m. Eastern. >> If you hold crypto on your phone, your biggest vulnerability isn't your wallet.

It's your carrier. AT&T, Verizon, and T-Mobile have been breached again and again. And SIM swaps are still one of the easiest ways for attackers to drain accounts. That's where Cape comes in, America's privacy-first mobile carrier. Same premium service, but Cape rotates the identifier on your SIM every 24 hours, deletes your call and text metadata after a day, and protects against SIM swaps with a 24-word recovery phrase that only you control. You also get two middle-to-end encrypted secondary numbers for banking and sign-ups. So, you stop handing your real number to every app that asks. Go to cape.co/unchained

and use code Unchained for 33% off your first 6 months.