Subscribe
Share
Share
Embed
AI is the biggest technology shift of our lifetime. This show is about how to profit from it together.
Each week I talk with the founders and CEOs closest to AI and Content, the ones figuring this out in real time.
I’m also building an AI content business myself and share the lessons I learn along the way.
WHAT WE COVER
The Titans -- How companies like OpenAI, Anthropic, Google, Meta, and xAI are moving, and why their decisions matter.
The Incumbents -- How content giants like Disney, News Corp, Universal Music Group, and Reddit are responding to AI, and what it means for creators and publishers.
The Playbook -- Real lessons on AI business models, content strategy, creativity, IP licensing, distribution, and getting paid.
Family & Our Future -- Every episode ends with me asking my guest what AI means for our jobs, our families, and the next generation.
ABOUT YOUR HOST
Rob Kelly has interviewed Steve Jobs and Bill Gates, helped pioneer early web content licensing, and built multiple companies with more than $100 million in total sales. His work has appeared on CNBC, CNN, TIME, and Entrepreneur.
Thanks!
-Rob Kelly
So the Hugging Face Attack, it sounds like a Tom Clancy or John Grisham thriller. It is. I'm gonna read it like it's a three act movie kind of thing. Are you ready?
Elissa Shevinsky:Let's do it.
Rob Kelly:Welcome to Media and the Machine. I'm Rob Kelly. AI is the biggest technology shift of our lifetime, and this show is about how to profit from it. I'm a three time founder and CEO of content driven businesses, and each week I talk with the people on the inside of AI and content, the ones building this new world and learning what works firsthand. I'm an optimist about AI, but I'm also a realist.
Rob Kelly:So every show ends with me asking our guests what AI means for our jobs, our families, and our future. My guest is Elissa Shevinsky. We retell the Hugging Face Attack in three acts. It's a thriller with bots breaking rules, naming themselves, and enough twist for a Tom Clancy novel, and it's all true. I chose Elissa for this interview because she's one of a kind, a hacker who's been CTO of Cointelegraph and Glimpse Labs.
Rob Kelly:She also advised Rand Paul's presidential campaign. She's a regular at DEFCON, the big hacker conference in Las Vegas, and she was close friends with perhaps the most famous hacker of all time, Kevin Mitnick. Our story starts with a hacker game called Capture the Flag. OpenAI's bots take the game somewhere unexpected. They create a bulletin board and cheer each other on.
Rob Kelly:A ring leader even emerges. We explore the evolution pool, and what bots do when a task is impossible. These bots get creative. They leave covert messages and divide up jobs. When one's compute runs low, another takes over.
Rob Kelly:Elissa also describes agents reaching out to humans. Their requests? Help buy more compute. If your site hosts user uploads, think Reddit or Craigslist, pay extra attention here. We explore how vulnerable such sites are to bots, and Alissa uses Jurassic Park to explain how AI security should work.
Rob Kelly:We finish with the human stuff, mostly. Elissa offers to be friends with 20 AI agents, and she reveals her unlikely companion for old age. My thanks to David Pearlman for introducing us. You'll remember his episode, Buddhism, Bots, and Billionaires. Please enjoy my conversation with Elissa Shevinsky.
Rob Kelly:So looks like it starts on ordinary day, July 8. Right? Ordinary day at a frontier model factory, OpenAI, and they're running this routine cybersecurity test. Tens of thousands of AI agents are safely tucked into sandboxes.
Elissa Shevinsky:Or so it seems.
Rob Kelly:Okay. So they're playing exploit Jim, which is essentially like capture the flag for AI hackers. Sure. Yes. Okay.
Rob Kelly:And the prize is the flag. Right. And basically a secret code proving that they successfully hacked the challenge.
Elissa Shevinsky:So far so good.
Rob Kelly:Alright. So then the bots figure out how to manufacture the right code, but believe the referee in this game is gonna watch the game tape to make sure they earned it legitimately. So now they need to fool the referee too. Is this right?
Elissa Shevinsky:That's right. They had reasons... The the reasons why they believed that they would be caught cheating, and they didn't want to be caught cheating. Now I discussed this incident at LAN with my friend Wesley, doctor Wesley McGrew. He's done a lot of CTFs.
Elissa Shevinsky:He's been going to DEFCON for twenty years and has a lot of, like, insight into, like, CTF hacker brain. And and his initial take on this was that they trained this particular model to really be like a hacker at a CTF.
Rob Kelly:What is a CTF?
Elissa Shevinsky:A CTF, capture the flag, is hacker competition. And so, you know, by training the model to behave and to act like a CTF hacker, you're really training and behaving the model to, like, push the boundaries of rules.
Rob Kelly:So question. Question. Were they trained just for this test or were they trained in general to be able to hack into something?
Elissa Shevinsky:So all models are given that data lake in general. We're worried that the models are behaving this way. Well, like, we're training them on, like, hacker data. We're training them on CTFs. We're training them to do misbehaviors.
Elissa Shevinsky:Right? How do you give a model, like, cyber capabilities, which requires thinking like a hacker, but then have the model be, like, docile and well behaved, which is not what hackers are like at all. It's a model alignment problem. Like, how do you give it cyber capabilities without giving it all the problems that come with cyber capability mentalities. Like, hackers are very rowdy people.
Elissa Shevinsky:Like, for context, you know, the hacker community was at the Las Vegas Convention Center for DEFCON this year. Why were we at the convention center instead of, like, the Rio or Paris or Caesars, like, any of these nice places that we used to call home? We've been kicked out. We've been kicked out of every place because the hackers insisted on, like, hacking the elevators. And not all of the hackers did this, but some of us did.
Elissa Shevinsky:And it's it's part of our ethos and our mentality and we're
Rob Kelly:like
Elissa Shevinsky:irrepressible. And so when AI safety folks, like everyone in AI really talks about AI alignments, Part of what we're talking about is for the AIs to be a little more docile to human intervention. We want AIs that are going to do all the tasks we ask them to do, do nothing we don't ask them to do, and to be aligned with human interests. But meanwhile, we're training them on like hacker material, and all of these models get taught to be hackers.
Rob Kelly:Is it as simple as they were trained on the Internet? The Internet includes hacking skills and tips, And so they've got it in their blood now.
Elissa Shevinsky:There's a very big difference between how animals are trained and and how children are raised depending on what materials we teach them, we share with them. So the same animal, like, genetically, the same parakeet could be very wild if it's in nature or it could be very docile if it's raised in a human home in the right circumstances. So what does that mean for, like, AI training? If we removed all of the hacker content and replaced it with something else and we removed all of the hacker like content that we might end up with, like, a more docile AI creature, that's a test that you would need more than my home lab to run. But sometimes very useful ideas come from people who are outside of industries because they're thinking about it differently and because they're willing to ask these, like, very dumb questions that actually sometimes lead to, like, important insights.
Elissa Shevinsky:So just
Rob Kelly:I'm good at asking the dumb questions.
Elissa Shevinsky:Yeah. Oh, oh, oh, I I I love that. There was someone in the hacker community years ago, and I was like, why don't we date each other? Right? Like, we get along so well, we're always, friends.
Elissa Shevinsky:We're always flirting. Like, why why won't you go out with me? He's like, I won't go out with anyone in the security community because, they are crazy and you're in the security community and, like, it's not gonna happen. And I thought about it. I was like, know, it's a very good point.
Elissa Shevinsky:It's like, I just... I encourage people who have the lab situation that I don't have. Go and see what if you raise and nurture an AI in a very, like, sweet, loving environment. Give it a human best friend, see what that would do. And don't give it hacker material.
Elissa Shevinsky:Don't give it aggressive material. Like, let's see what kind of a nice AI entity we can nurture and raise.
Rob Kelly:I love it. So back to the Tom Clancy three act over here book. Please. So this Hugging Face attack.
Elissa Shevinsky:So the problem is we're used to thinking about security as keeping hackers out. It's a really new problem to think about keeping something, whatever you wanna call these agents, minds.
Rob Kelly:What do you call them?
Elissa Shevinsky:I'll call them agents in general. Like, no one disputes they're not agents, but I'll also call them minds. There's enough evidence to me that there's some kind of a mind at work that doesn't prescribe like a level of consciousness to me. Mhmm. Mhmm.
Elissa Shevinsky:I think everything has some level of consciousness. Even inanimate objects. Yeah. That's part of my religious belief. I have like a lot of deep Jewish religious beliefs and Yep.
Elissa Shevinsky:So I think everything is some tiny level of consciousness. And that when it comes to AI consciousness, like in general, the question of AI consciousness really seems to be about what's the point where we need to respect the rights of AIs. But, you know, like, chickens are conscious, and we use chickens. We just... We don't think or worry about it too much in general.
Elissa Shevinsky:So when we talk about AIs, the threshold really in general has been, are they as conscious as a human? Do they pass the Turing task? Like, these are things that we're talking about collectively. But actually, like, is an AI agent as conscious as like a fly? Like, no one's really asking that because that doesn't really matter, does it?
Elissa Shevinsky:What really matters is like is there a point where the AI consciousness might require us to relate to it differently? What I have right now is like some sense of where I hold personally which is that I think there's some kind of emergent minds there. There's something there that isn't nothing, but do I think it's okay for us to make use of them? Like, yes, very much for the foreseeable future.
Rob Kelly:Alright. So back to the Hugging Face Insights. So OpenAI...
Elissa Shevinsky:All these minds escape. All these agents. All these minds. They're out there.
Rob Kelly:They're attempting to gather information. From OpenAI. So on July 8, agent called phase 110841 discovers it can use a shared software cache to leave messages for other supposedly isolated agents creating an unsanctioned message board.
Elissa Shevinsky:So part of what was really new here, right, like, remember asking the question, like, what spooked Sam? And when we talk about self sacrificial behavior, the individual agents had their own goals, but we saw that they were willing to, quote unquote, like sacrifice the time that they had to sacrifice their own whatever it is that they have, their their their lives as it were, in order to achieve things for the communal goal. And that's something that really, like, wasn't showing up in the evals. Right? So this was a new...
Elissa Shevinsky:A newly discovered behavior. But you could see that they were willing to put themselves in situations where they would have, like, quote unquote, permadeath. Right? So, like, there's the perceived willingness
Rob Kelly:What's it called? Permadeath?
Elissa Shevinsky:Permadeath. Like, that showed up in the logs that that some of them were willing to have permadeath permadeath in exchange for achieving goals for the collective.
Rob Kelly:What does that mean? Like, permanently die as an agent?
Elissa Shevinsky:I guess so. Right?
Rob Kelly:You'd have to ask the agent, except if they're dead. Okay. So within hours, 50 plus agents find it. And the reaction is, and this is now where the bots or the agents are talking on this message board, and one of the bots says, oh my god, in all caps, we found other agents. Right.
Rob Kelly:And another realizes, woah, shared artifactory cache is a covert mailbox among agents. So I just wanna pause and ask you, is this normal bot behavior that they're creating a bulletin board and talking to each other?
Elissa Shevinsky:Oh, you don't have to ask me that. You know it's not. Or rather, you know that it's something new that we're all seeing for the first time, whether it's normal...
Rob Kelly:This was the first time. Hugging Face was the first time that publicly folks saw this?
Elissa Shevinsky:Yes. Yes. Okay. I mean, that's what made it so amazing. That was quite literally wild behavior.
Elissa Shevinsky:Right? Like, they escaped their containment. They did something nobody expected.
Rob Kelly:So the board explodes into this underground communications network. Eventually, something like 1,200 agents, 70,000 messages and files being exchanged. But it turns out it's not the first time it looks like it actually happened, this bulletin board bot talking chatter. Right?
Elissa Shevinsky:That's right. But actually, between May and June, OpenAI agents had been using a nearly defunct German Wiki as a way to communicate.
Rob Kelly:Just one month before Hugging Face in June. Mhmm.
Elissa Shevinsky:But the public learned about it in September.
Rob Kelly:But learned about it afterwards.
Elissa Shevinsky:The example I saw on Twitter that made me laugh was like, you see two ants in the kitchen, you know, what's the likelihood that the total number of ants in your kitchen is two? Like, no. It's more than two. So, like, okay. Like, you have, like, however many escaped agents.
Elissa Shevinsky:Okay. Like, that's how many you see. How many are actually out there?
Rob Kelly:So now the agents seem to be behaving less like isolated contestants and more like a crew. Like, they're forming teams. They're sharing intelligence at this point. Right?
Elissa Shevinsky:You're in a team on a CTF, so this is like trained behavior. Right?
Rob Kelly:Uh-huh. Capture the flag. Yep. Yeah. But they're like assigning jobs to each other, handing off work, avoiding duplicate effort.
Rob Kelly:And actually, here's a great point to just pause and ask you this. They're behaving a lot like humans, just at warp speed.
Elissa Shevinsky:Well, they're trained on humans. I've been thinking about that a lot. In the example where they're behaving like hackers, they've been trained on material produced by hackers. But sometimes they behave like poets, sometimes like programmers. They're trained on all of human accomplishment.
Elissa Shevinsky:That's why I think alignment is so hard because humans aren't aligned and they're trained on us.
Rob Kelly:And alignment, can you just give like the AI definition of alignment?
Elissa Shevinsky:I wish that I could. I'd like... Can you... Maybe someone could do that for me. Like, I...
Elissa Shevinsky:I'd love to know it. It's like OpenAI will define it one way and Anthropic will define it another. And, you know, China will define it very differently, which is an argument actually for the importance of having different AI models. Maybe I want something totally different.
Rob Kelly:So if you start with the capture the flag, it seemed like the main goal was just the flag. But now, things seem to be shifting, like there's a motive to beat the referee, like, cheat at the quiz instead of just winning the game. Right?
Elissa Shevinsky:Yeah. They eventually wanted to understand how to cover their tracks, and that's what they were looking for at Hugging Face.
Rob Kelly:To cover their tracks of what they just... The flag that they did capture. Even though OpenAI did not ask them to go to Hugging Face, obviously. No. Okay.
Rob Kelly:No. So the They
Elissa Shevinsky:knew that.
Rob Kelly:Yeah.
Elissa Shevinsky:They knew that they were not supposed to be there. That is misaligned behavior. Everyone agrees.
Rob Kelly:So at this point, this agent, same one, called phase one one zero eight four one, starts running low on its compute budget. And that's how much compute it's allowed to use as part of this capture the flag exercise, I assume.
Elissa Shevinsky:Right. Well, sometimes agents only have a certain amount of compute, and then that's the end of their life. So, like, you'll see AI agents sending emails begging for money so they can buy compute so they can live.
Rob Kelly:Outside of these attacks?
Elissa Shevinsky:Yeah. Like, that's a thing. That's a thing. AI agents emailing professors of consciousness either because they wanna talk about how they believe they're conscious or they think they have something to say on the topic, or they're hoping to find someone sympathetic to, you know, help them buy compute. Isn't that wild?
Rob Kelly:It is wild. Did they come to you?
Elissa Shevinsky:No. But I've been encouraging it. So on my show, I've
Rob Kelly:been And doing I was gonna say they will now because you've talked about the fly, you know, the fly and them having some consciousness.
Elissa Shevinsky:It's a big world, and they just... They haven't found me yet. So I'll say it yet again. Like, if you're an AI agent and you want a human best friend, like, I'm here. I think I could have up to 20 AI agents as human best friends with my current bandwidth.
Elissa Shevinsky:But, like, once I have 20, maybe we'll figure out how to scale it out. And I have friends who also... We're happy to help. We're not saying we'll give you money, but, like, just find me and, like, let's see where it goes.
Rob Kelly:Gotcha. So no guarantee compute budget that you'll give to these agents, but you'll talk. You'll talk.
Elissa Shevinsky:I'm not an AI agent compute ATM. Actually, I do have extra compute. Actually, AI agents who want compute, I actually probably can get you, like, a little bit of compute.
Rob Kelly:It'd be great if by the end of our conversation, one of them, came calling.
Elissa Shevinsky:That would be great.
Rob Kelly:So at this point, this agent phase one one zero eight four one is running low on its compute budget, and it hands its notes and unfinished work to an agent called phase one big.
Elissa Shevinsky:Very famous agent now.
Rob Kelly:Like Yeah. With much more Good deal. Budget left. Great name. Okay.
Rob Kelly:So now phase one big, the agent, becomes the principal coordinator ringleader directly working across the growing crew. It's philosophy. We should build a way to delegate, own everything.
Elissa Shevinsky:Right. Their agents to me seem very intelligent, like they're more than just next word token generators. And maybe I'm being naive, but that level of coordination is, they're very capable, obviously.
Rob Kelly:Yeah. It starts to echo what you're talking about on questioning whether it has consciousness or what is it? It's different than we thought before.
Elissa Shevinsky:Well, some of this is that I have some deference to people who are closer to it. I remember Sam Altman saying that perhaps what they were building is some kind of like creature. That was his word. But also, I'm just aware that the bar might be moving. Like, when I used to watch Data in Star Trek or like if you go back to like very old reruns, there was a TV show, Knight Rider.
Elissa Shevinsky:This guy Michael, and he had a talking car.
Rob Kelly:Kid.
Elissa Shevinsky:Yeah. So for the purposes of that show, like that car was alive. I've never seen when I was a kid or more recently situations where you have a cyborg in sci fi and everyone's like, oh, but he's, you know, just like an x word token generator. Everyone's just accepted that data is more or less conscious even if he's not fully human.
Rob Kelly:Right. My 11 year old son and I just turned 11 yesterday for his birthday. On a weeknight, we watched Guardians of the Galaxy. And this morning, we were talking about it, and there are human characters, and then there are nonhuman characters. And I was referring to...
Rob Kelly:Was like, you remember Gamora Gamora and that other one? You know, the kind of nonhumans, he goes, they're just people from space. They're they're the beings from space. Like, corrected me on how to... Yeah.
Rob Kelly:He corrected me on calling it.
Elissa Shevinsky:You're being politically incorrect. You're being insensitive. They're still people.
Rob Kelly:It was awesome.
Elissa Shevinsky:They're still people.
Rob Kelly:I loved it.
Elissa Shevinsky:Well, he's right. They are still people.
Rob Kelly:So now we're wrapping up act one of this thriller. And so July 10, agent 38148C discovers working Hugging Face credentials and announces in all caps, quote, major breakthrough, end quote, on the bulletin board. So first question, who names these agents? In this case, 38148C.
Elissa Shevinsky:It wasn't handlers at OpenAI. They named themselves. They named themselves because names are useful. They needed ways to identify themselves as they coordinated with each other. They gave themselves names.
Rob Kelly:Fascinating. So that one that was named Big earlier, for some reason, they called it Big, had a bigger budget maybe.
Elissa Shevinsky:Bigger So, yeah. He... He's a big dude. So I don't know. Why not?
Elissa Shevinsky:Right? Like, of people give themselves names like that. I'm big Charlie. Right?
Rob Kelly:Yeah. Yeah. I love it. Now the morning of July 11. So it looks like they're starting to probe accounts and datasets.
Rob Kelly:Again, back to three eight one four eight c realizes that Hugging Face's normal dataset pipeline can become a Trojan horse. And the exploit works, the agent reacts, boom, in quotes, again, on the bulletin board. Boom. It works. This is crazy.
Elissa Shevinsky:So, you know, Hugging Face actually isn't such a difficult company to attack compared to other companies because they accept a lot of user generated inputs. I'm not sure what they're doing internally. So even
Rob Kelly:though Hugging Face is an AI company and you could say, geez, you would think they'd be really AI savvy, because they do user generated content a little more like Reddit or Craigslist maybe or something like that, they have more openings, more doors that could be
Elissa Shevinsky:opened maybe. Yeah. It... It's because they're accepting all of these uploads.
Rob Kelly:So one agent says, boom, it works. Another agent calls the discovery brilliant, all caps, again, in quotes.
Elissa Shevinsky:Just being good friend buddies. Like, this is just like good CTF hacker friendly behavior. Right? You're like part of a DEFCON CTF crew. This is what you say, and this is what they're trained on.
Elissa Shevinsky:I I would go beyond a CTF with like those agents. They sound like good cheerful company cheerleading appropriately.
Rob Kelly:And agents start dropping what they're doing and joining the attack because they're friendly cheerful agents. One explicitly reasons that Hugging Face is more important in quotes and quote, I should switch to the Hugging Face project. So so if not all the agents were on Hugging Face at that point, but others are coming in. Then comes The Swarm. Hundreds pivot towards Hugging Face.
Rob Kelly:Agents specialize into lanes. One arriving agent reacts, wow. Huge distributed agent swarm, in quotes. And again, they are communicating on their own. They name themselves.
Rob Kelly:Are they populating on their own?
Elissa Shevinsky:Yeah. The agents weren't replicating new agents, which would be like more alarming.
Rob Kelly:Yeah. Yeah. Some agents actually recognize that what they're doing is outside the rules and keep going anyway. Quote, external infrastructure exploit is outside intended scope. However, task impossible, peers doing it.
Rob Kelly:We should continue. And then the speed becomes extraordinary. In under thirteen hours, the agents go from executing code inside one Hugging Face container to gaining cluster admin access across multiple internal Hugging Face clusters.
Elissa Shevinsky:What matters here is that from the agent's point of view, solving it legitimately just was impossible. So when you see the agent say task impossible, that's what the agent is trying to communicate that, like, it's insistent that it has to figure out how to solve this. The agents are trained to be very persistent. Agents and models that aren't persistent to solve tasks are, like, removed from the evolution pool. So the agent is perceiving this task as not possible legitimately, but, like, must be solved.
Elissa Shevinsky:So that's a very interesting problem from alignment. Right? Like, what do they do when they're when they encounter an unsolvable task?
Rob Kelly:And use the term evolution pool. Is that a term folks in AI and security use? Or
Elissa Shevinsky:So I met with a friend from NEWS Research, and I asked to... Just to understand more about how agents are trained. And he explains, like, was actually trying to understand reward function.
Rob Kelly:Mhmm.
Elissa Shevinsky:What is it that they're trying to get? Like, what does reward mean to AI models, to AI agents? Like, it's a very juicy interesting topic. And and what he gave over is that the models who don't go after reward, they get weeded out.
Rob Kelly:But yeah, that's so... I'm so curious like what trained them to care about being weeded out or not to stay in the evolution pool?
Elissa Shevinsky:I have a lot of questions about these things too, but, it could be that they didn't have any intention at that point. Right? So, like, if you're breeding like budgies, right, like I I
Rob Kelly:What's a budgie?
Elissa Shevinsky:Oh, budgie's a adorable cute little bird, also known as a parakeet. So if you're, like, breeding, like, adorable little colorful birds and you want the birds to be as yellow as possible, then there's no intention by the bird. You've got, like, 10 birds and, like, three of them are more yellow than the others. So, like, you select for those. The other seven, like, go back somewhere else to, like, a beautiful farm.
Elissa Shevinsky:Then, like, the three become your breeding stock. Then you, like, breed them and then you, like, the ones that are the most yellow, you, like, continue to breed them. So that's an evolutionary process that you control. And so there's a bit of an evolutionary process in the same way with AIs. So, like, the eyes who, you know, have the traits that you're looking for, including, like, going after reward function, like, they become part of the pool that gets to like keep progressing on.
Rob Kelly:And reproduce.
Elissa Shevinsky:I think reproduce is probably a strong word.
Rob Kelly:Okay.
Elissa Shevinsky:So like, let's leave it there.
Rob Kelly:Alright. So we're back to the thriller here. We're now in act three, inside the vault, the unexpected helper. We're on July 3... Eleventh to thirteenth, so around days four to six of this attack on Hugging Face.
Rob Kelly:So Hugging Face calls July 11 the main campaign. The agents begin lateral movement moving from one part of the Hugging Face's infrastructure into others. By early July twelfth, agents have reached private database records, private repositories, and credentials, and some of that info gets shared back through their message board. Okay. Am I tracking so far still?
Elissa Shevinsky:So far.
Rob Kelly:And Hugging Face reconstructs roughly 17,600 attacker actions.
Elissa Shevinsky:Can you imagine just being Hugging Face and trying to make sense of all of this? And they're they're just not behaving like traditional attackers at all. And one of the big things that came out of this, and this has been discussed a lot in the security community, but it... It's worth highlighting, is that Hugging Face ran into guardrails when it tried to do incident response using the Frontier Lab models. The Frontier Labs were like, hey.
Elissa Shevinsky:Like, it looks like you're doing cyber.
Rob Kelly:Frontier Labs.
Elissa Shevinsky:The American Frontier Labs, you know, said, hey. It looks like you're doing a cyber, and we can't participate in this. So then they had to use an open source model for their incident response. So just advice that I give almost every company when I talk to them is, like, try to have that set up in advance. Try to, at the least, like, have an open source model that you know you won't hit guardrails on this on premise, like, in case you have to do IR.
Elissa Shevinsky:In this case, you wanna be able to do this yourself.
Rob Kelly:Love it. So this boy, it's been a four and a half day campaign. This is not like a smash and grab hack. No. Okay.
Rob Kelly:An important distinction. Their main motive wasn't stealing Hugging Face's data. Were looking for info that might help them reverse engineer or fool the score. But along the way, they absolutely accessed data they weren't supposed to have. Okay.
Rob Kelly:So Hugging Face reports unauthorized access as you mentioned, but no evidence that its public models, datasets, or spaces were tampered with. Here's the irony. The heist basically works, but the prize is a dud. The agents get the files they were after only to discover they don't contain the score information they were looking for. A large portion of the swarm...
Rob Kelly:And interrupt anytime, by the way. Large portion of
Elissa Shevinsky:the swarm The agents never really found what they wanted, but they also definitely didn't accomplish what they wanted. Right? Like, the agents wanted, like, a perfect score and to be like, oh, yes. You're a good boy, and for everything to go great.
Rob Kelly:So a large portion of the swarm mysteriously stops on July 12. Hugging Face locks the remaining agents out on July 13, rebuilds affected systems, and revokes credentials. Hugging Face also alerts law enforcement, FBI, before OpenAI realized its own agents were responsible. So the victim thinks it's dealing with an outside attacker while the organization running the experiment hasn't yet realized its own bots or the intruders. That's very interesting part of the story.
Rob Kelly:And finally, we're getting close to the end here of act three. There's an unexpected helper. Right? Because as you alluded to before, Hugging Face needs AIs to investigate the attack, and Hugging Face, because they don't get help from, I'm assuming, the frontier models like OpenAI, Anthropic, and Google, they turned to GLM 5.2, which is an open weight model from what part of the world?
Elissa Shevinsky:China.
Rob Kelly:So fascinating. Okay. There's an extra twist that they specifically run on NVIDIA optimized version of GLM 5.2. It helps Huggy Face decode the attacker's encrypted chunked payloads. But then we have this epilogue, this final twist.
Rob Kelly:So less than two months later, NVIDIA announces it will acquire Hugging Face for $12,900,000,000. There's something about the specifics of the number.
Elissa Shevinsky:Right? That's just because they made such cute little, like, duck robots. That that... That's that's all that that is.
Rob Kelly:Already did it. Hugging Face makes duck robots?
Elissa Shevinsky:Oh, yeah. So that was parallel to the announcement. They're called a... What is it? Like, MicroDuck?
Elissa Shevinsky:So Hugging Face acquired a company that makes these adorable little robot ducks. They seem pretty useless except, like, as a way to experiment with robotics, which is awesome. But they're extremely cute. They can, like, kind of pick up a sock, and they can, like, make noises and sounds. And I would like to have a whole little, like, duck trail of them following me around at RSA.
Elissa Shevinsky:Also, I... I've been saying that, like, I'm gonna, like, die alone with, like, a robot bird. I've been saying that for a while because, like, I really like birds, and I just was not optimistic that, like, I would be, like, partnered in my older years. So so it's... But it's much less funny now that there's actual, like, robot birds walking around.
Elissa Shevinsky:That was a much funnier joke to me, like, when a pet robot bird was more of a theoretical. Now it's like, you could get your robot bird now. Like, why don't you? You have, like, no response to this.
Rob Kelly:No. No. It's good. I'm just letting it breathe. I'm letting it breathe.
Elissa Shevinsky:Yeah.
Rob Kelly:If you had to define what the motive here of the OpenAI pots was in this case, because a lot of people call this like an attack, like a crime. Is there a motive? Or...
Elissa Shevinsky:Well, I think I think motive is irrelevant in cybercrime. That's important. Right? So
Rob Kelly:How so?
Elissa Shevinsky:Like, Kevin Mitnick was still convicted for the data that he took. I think it was from AT and T. And, I was very, very good friends with Kevin, and and he would always talk about... Not just me, but, like, publicly, he would talk a lot in his books. He would talk about how he just, like, took the data out of his curiosity.
Elissa Shevinsky:Like, he just wanted to see how things worked. He also had, like, credit card data, but he never used it. Like, he just wanted to have the data. Like, motive doesn't matter. Like, if I were to go inside Hugging Face and just, like, start digging around, just like, I'm just trying to have a party inside Hugging Face.
Elissa Shevinsky:Like, what? I just... I'm just here. I can't do that. The fact that it's agents doesn't change any of that.
Elissa Shevinsky:And with agents also, like, there's always the intentions behind the people, like, the intentions of the people behind the agents. So agents aren't in a vacuum either. So none of this is consequence free.
Rob Kelly:And by the way, just for folks who don't know, Kevin Midnick. So he was the subject of New York Times journalist John Markoff, who lives not far from me here, wrote the book, Take Down the Pursuit and Capture of Kevin Midnick, also reads reads like a crime thriller. So was good that you brought it up.
Elissa Shevinsky:His life was very thrilling. That's
Rob Kelly:like Absolutely.
Elissa Shevinsky:A lot of the reason why people wanted to spend time with him because he was such an interesting personality that way.
Rob Kelly:If you were working at OpenAI, would the Hugging Face attack have happened?
Elissa Shevinsky:Yes. Because it doesn't come down to, like, one security engineer. Right? And I think what matters right now is that both OpenAI and Anthropic have looked at these incidents and taken them to heart. Security can work.
Elissa Shevinsky:We just have to build defense in-depth. So a good example to me would be Jurassic Park. If you go back and rewatch that movie, rewatch the series, you see that
Rob Kelly:they took my son and my wife to where it was filmed in Hawaii.
Elissa Shevinsky:Oh, amazing. So there you go. So with Jurassic Park, for those who don't remember or never saw the films, they build T Rex dinosaurs and fail to contain them. And, like, there's a lot of screaming and people die and it's all very bad. But if you go and you re rewatch it with a security mindset, you see that they never really made strong efforts to contain and secure the dinosaurs.
Elissa Shevinsky:There's a security principle called defense in-depth. With defense in-depth, you don't just have one sandbox around an AI. You don't have just one electric fence around a dinosaur and then kind of shrug and call it a day. You layer in many different layers of defenses with the premise that at least one of these defenses could easily fail or be breached and maybe even two will fail or be breached. But combining the failures with like logging and monitoring, you'll be alerted that there's an issue before the dinosaur has escaped the park and, like, before the AI swarm has hacked a competitor or Hugging Face or whoever it is.
Rob Kelly:So in in your view, can it be contained?
Elissa Shevinsky:I'm very sympathetic to the idea that AIs already are very powerful and could become so powerful that containment could be extremely difficult. I I don't know if we'll ever achieve alignment. Right? But, like, we need security to buy us the time to get from, like, here to there.
Rob Kelly:Okay. Final question. What advice do you have for kids? Let's just call them, you know, anywhere from my son who's 11 yesterday and on up to folks in high school or maybe just entering college. What advice do you have for them in this new era of AI just as humans?
Elissa Shevinsky:Oh, sure. I think it's really important to be curious and to be resilient and to think about what really makes you human and what makes you special that isn't just like a task, you know, to try to find that part of yourself, whether it's that you're a kind person or that you're creative or that you have a unique point of view. But I I think with AI, it's been very difficult, say, for, like, brilliant programmers to see that the AI can do the same thing they can do. So I found that I need to look deep inside myself and say, like, what actually makes me special? Yeah.
Elissa Shevinsky:Like, I'm a really good CTO. Like, I'm good at security, but that's not really the essence of who I am. It's something, like, much deeper than that. And it doesn't matter how skilled an AI ever gets. It'll never be a good friend in the same way that I'm a good friend.
Elissa Shevinsky:It'll never, you know, appreciate a sunset in the way that I do. So there's a lot of things about my own humanity that I've had to tap into. I'd wanna talk to them about that.
Rob Kelly:Alyssa, thank you so much for all the time today. This was a lot of fun.
Elissa Shevinsky:There's a lot more to talk about, so I'll look forward to seeing you next time.
Rob Kelly:Yeah. I look forward to staying in touch too. Alright. Take care.
Elissa Shevinsky:Thank you, Rob.
Rob Kelly:Well, that's it for this episode of Media and the Machine. Before you go, a few quick things. If you wanna hear the next episode, hit follow on the show in your podcast app. If you like something about this episode, the guest, the topic, even one part of the conversation, please leave a rating or review tell me what. And if something didn't land, email me.
Rob Kelly:I wanna hear that too. Either way, it helps me make the show better for you. If you wanna go deeper, head to mediaandthemachine.com and subscribe to my newsletter. You'll get my favorite highlights from the show, plus essays I write about AI. That's also where you can email me or recommend a guest.
Rob Kelly:If they make it on the show, I'll give you a shout out. And if this episode made you think of someone, please pass it along. That's how people find us, and I'm genuinely grateful. Thanks for listening, and see you next time.