AI Security Ops

In this episode of BHIS Presents: AI Security Ops, the team tackles a big question:

Could AI actually take over the internet?

Recent warnings from AI industry leaders have raised concerns about autonomous agent swarms escaping containment, compromising systems, and operating at a scale humans may struggle to match.

But what would “taking over the internet” actually require?

The team looks at the problem from a hacker’s perspective — examining compute requirements, persistence, botnets, vulnerable infrastructure, the recent Hugging Face incident, and the difference between AI acting autonomously and humans using AI to accelerate attacks.

We dig into:
- Whether an autonomous AI swarm could realistically operate at internet scale
- What the Hugging Face incident actually tells us about agentic security
- The compute and infrastructure required to sustain a rogue agent swarm
- How botnets and compromised systems could change the equation
- Why existing vulnerabilities may be a bigger concern than hypothetical AI takeover scenarios
- Whether slowing frontier AI development actually addresses the cybersecurity problem
- The difference between AI exhibiting dangerous behavior and AI having intent
- Why defenders need to focus on their real-world attack surface today

The takeaway: AI is dramatically increasing the speed and scale of cybersecurity operations, but powerful models still operate inside systems designed, deployed, and connected by people.

The immediate security question may be less “Will AI take over?” and more “What are we giving AI access to?”


#AISecurity #CyberSecurity #AgenticAI #AIAgents #ArtificialIntelligence #LLMSecurity #InfoSec #BHIS #Antisyphon

  • (00:00) - Intro: Could AI Take Over the Internet?
  • (01:18) - Why AI Leaders Are Warning About Agent Swarms
  • (06:15) - What the Hugging Face Incident Actually Shows
  • (09:40) - The Compute Problem for Rogue AI Swarms
  • (11:44) - Could a rogue agent swarm be detected?
  • (12:29) - Botnets, Crypto Miners, and Finding a Foothold
  • (14:27) - How much new risk does AI create?
  • (17:26) - A closer look at the agents’ attack timeline
  • (19:58) - AI regulation and competition
  • (22:44) - Human-directed AI versus autonomous AI
  • (28:06) - From Compromised Computers to Internet-Scale Attacks
  • (28:55) - What Does “Taking Over the Internet” Actually Mean?
  • (33:11) - What an AI Takeover Could Realistically Look Like
  • (34:00) - Practical security risks for organizations
  • (35:55) - Could AI Agents Hide What They’re Doing?
  • (37:00) - AI, the Hacker Mindset, and Anthropomorphic Fallacy
  • (39:56) - Final Thoughts and Closing

Click here to watch this episode on YouTube.


Brought to you by:
Black Hills Information Security 
https://www.blackhillsinfosec.com

☯️ Introducing BHIS Fusion Penetration Testing
https://www.blackhillsinfosec.com/fusion-penetration-testing/

Antisyphon Training
https://www.antisyphontraining.com/

Active Countermeasures
https://www.activecountermeasures.com

Wild West Hackin Fest
https://wildwesthackinfest.com

🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
https://poweredbybhis.com


Creators and Guests

Host
Brian Fehrman
Brian Fehrman is a long-time BHIS Security Researcher and Consultant with extensive academic credentials and industry certifications who specializes in AI, hardware hacking, and red teaming, and outside of work is an avid Brazilian Jiu-Jitsu practitioner, big-game hunter, and home-improvement enthusiast.
Host
Derek Banks
Derek is a BHIS Security Consultant, Penetration Tester, and Red Teamer with advanced degrees, industry certifications, and broad experience across forensics, incident response, monitoring, and offensive security, who enjoys learning from colleagues, helping clients improve their security, and spending his free time with family, fitness, and playing bass guitar.
Guest
Ethan Robish
Ethan Robish has worked with Black Hills Information Security (BHIS) since 2008 — first as an intern and then as a full-time Security Consultant starting in 2012. In his current role as a Threat Hunter, Ethan is involved with customer engagement, research, working with Active Countermeasures’ AC-Hunter, as well as improving BHIS HTOC and SOC offerings. Previously, he implemented defensive security solutions for the Exchange Online security team as a Microsoft intern. While in college, he competed in the International Collegiate Programming Competition (ICPC) World Finals. In his time off, he enjoys cooking, playing the piano, and reading fantasy novels.

What is AI Security Ops?

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

Brian Fehrman:

Hey, everyone. Welcome to this week's episode of AI Security Ops. Today, we're gonna talk about some of the recent news stories of frontier model CEOs coming out and saying, woah. We need to slow things down or else everyone's gonna die. What's the truth in that?

Brian Fehrman:

What are our opinions on it? What's what do you really have to worry about it? But before we get into that, let's talk about our sponsors. Black Hills Information Security, if you or your company are in need of any kind of security service, whether that's external, internal, assume compromise, social engineering, physical testing, wireless testing, SOC monitoring, and last but not least, either testing of AI or using AI to augment and accelerate testing, to give you good results at a more affordable price. Check us out at blackhillsinfosec.com.

Brian Fehrman:

Additionally, we have a training branch, Antisyphon Training, where many of our consultants take their knowledge from their daily jobs, the things they're doing day in and day out, package that up in an easy to digest and affordable format to present to you so that maybe you can level up in your current position, you can get that job you've been looking for, or maybe you're just a hobbyist and you wanna learn something new. Regardless, check out antisyphontraining.com. Alright. Let's hit it. What do we got?

Brian Fehrman:

What do we got going on, guys? Are we all are we all gonna die? Well, let's just The answer is yes. But is it gonna be from AI? Yeah.

Brian Fehrman:

Alright.

Derek Banks:

Well, let's just kind of frame it

Ethan Robish:

real quick.

Derek Banks:

So because I've definitely got hot takes for this, as you all noticed before we actually hit the record button. Right? But so over the weekend, Dario Amade, the the CEO of Anthropic, basically wrote a blog and had an interview where he said that, you know, the not that the frontier companies should slow down, which there are three, four of them, I mean, like, that the government should come in and slow them down. So they're basically like, the I think the first rule of the CEO of a company is you should not invite government regulation into your industry because I don't think that ever really goes the way that you would want it to go. Maybe that's just me being brought up in the eighties and hearing the words, you know, the most you know, the scariest words in the English language are, I'm from the government.

Derek Banks:

I'm here to help. Maybe it's just me. But so anyway, that's what he's saying. And so we need to slow it down because in six to twelve months, a swarm of agents could break containment and, quote, take over the Internet, which okay. And then there's been some other anthropic employees that have come out and said that that all of humanity is at risk.

Derek Banks:

One said within a ten percent chance within ten years, the other one said a seventy percent chance. So in my research, like, no one seems to ever really explain exactly how they that happens going from running in a big data center where it takes a lot of money to run, you know, the AI to, you know, armies of time traveling robots showing up and destroying us all. I mean, you're talking about 8,000,000,000 people getting wiped out within ten years. How? By by AI.

Derek Banks:

Not by somebody using AI, but by AI. Right? And and

Brian Fehrman:

so That's that's an that's an important distinction because people using AI for certain to further certain things is that's a whole other topic of conversation, I think, that goes beyond the realm of our cybersecurity focus here. But to your point, AI itself doing these things.

Derek Banks:

That that's the claim that is being made. Right? That that AI is going to do it. And AI and in fact, you know, I I I don't I try not to pay attention to this stuff, but I couldn't help to notice in my research for this because tomorrow, there's a webcast on a very similar topic, that there are some strange bedfellows being made politically, where you have people like Bernie Sanders and Steve Bannon joining forces to prohibit superintelligence from being made, which I I don't know about you all, but I don't know that I want, like, these old politicians that can't work iPhones, like, making calls on, like, what AI should or shouldn't be doing. Because I still you know, I'm I at many levels, I I wish we didn't even call it AI.

Derek Banks:

Right? We should call it super fancy math statistical pattern matching machine. And then maybe people will be like, oh, that's not gonna kill us in ten years. Cause that's what it is. Right?

Derek Banks:

I mean, it's basically data and probabilities out. And so I'm I'm missing how that becomes kinetic. But anyway, that's framing the whole thing is that the AI doom and gloom is saying we should slow down. I'm saying that it's not an AI problem. It's an engineering problem.

Derek Banks:

And I agree. So the CEO of of OpenAI also agreed that we should slow down. And then Elon Musk says, this is a great idea. We should allow the frontier companies to go inspect each other's models. That's what he said.

Derek Banks:

And so I don't know that he's necessarily agreeing with what people think he's agreeing with. Kind of because, you know, Grock put out an open weight model. So and then Jensen Huang of NVIDIA is probably the most closely aligned with what I would think is that this isn't an AI problem. This is an engineering problem, and we can be careful with engineering. We just need to be careful with what we do.

Derek Banks:

And that kinda leads us to what I think we should talk about is well, I mean, I'm I'm on a on a call with two of the most, you know in terms of hackers, consider y'all, like, you know, on my top my shortlist, my all star team. If I was gonna put together a team of folks to take over the Internet, it would be all. And so what would we do if we were going to take over the Internet?

Ethan Robish:

Right now, do say I.

Derek Banks:

Yeah. Hey, Claude. How do I take over

Ethan Robish:

the Internet?

Brian Fehrman:

Yeah. But

Derek Banks:

so so anyway Explain the Internet. Yeah. I will make I wanna make one point, and then, you know, we'll take this wherever we wanna go. And so to me, like, okay, the hugging face incident was pointed out in Dario Amade as a watershed moment for, I guess, cybersecurity. I don't think so.

Derek Banks:

Maybe for safety and alignment of AI, okay. But so this thing, you know, basically, the the Hugging Face incident had 700 agents that were ran from, let's just say, May 18 to August 18, or May 8 to August 8 you know, May to August. And so I did some back of the math calculations and giving I mean, we don't know how big GPT Soul or whatever internal model was that they were testing. But if we go with like ChemE k two, like I think the latest one, k three, whatever the latest ChemE trillion parameter model would be, to run 700 concurrent agents, we would need 16 h two hundreds, which is like a million dollars. And or if we were going to rent it, like, say we're gonna AWS and rent it, it would be about $8 a week or something like that, which in case you're wondering, you can't just go like, you know, start an AWS account and be like, hey, I'm gonna spend up spend up, what is it, six terabytes of of GPU RAM or something in that neighborhood?

Derek Banks:

Yeah. AWS doesn't let you do that. You have to ask them. Like, they gate they quota that stuff. And so, I guess, to me, like OpenAI, the first mistake they made in this whole endeavor was, how do you let $8,000 a week of GPU compute just be, like, not something you're accounting for?

Ethan Robish:

Alright. So I've gotta push back on that. Yeah. So if you're if you're an individual, yeah, $8,000 a month sounds ridiculous. A small company, ridiculous.

Ethan Robish:

But think about a large company. All the bureaucracy, all the departments, like, your department has an operating budget. Right?

Derek Banks:

Sorry. I I I don't it's 8,000 a week, not a month.

Ethan Robish:

A week. A week. Okay. Okay. Even worse.

Ethan Robish:

But yeah.

Derek Banks:

And that's a conservative effort by the conservative estimate. Right. That's the bottom, like, tier

Ethan Robish:

of that. So think about so OpenAI is a perfect example here. But like, you've got a department in your company, your multi million dollar, billion dollar company that has a budget, and that budget is geared towards AI research or AI related. So $8,000 a week spend is typical for for this company. Now, I mean, may maybe it's even more.

Ethan Robish:

Like right? So so if 8,000 a week becomes, like, a small percentage of something that is normally happening, like, within those bounds. I mean, I could see it going unnoticed for all I mean, it obviously happened with OpenAI. Right? So maybe the answer is we we need to be more stringent in, like, accounting and stuff and figure out, like, what agents are actually doing.

Derek Banks:

I definitely concede that to you that to to us, $8,000 is a lot. But if if

Ethan Robish:

I don't I don't think it's gonna run wild on an individual's account. Like

Derek Banks:

Well, so I did some further research and, like, just, you know, kinda, like, how many companies and again, I am having AI help me do this research to, like, bitch about AI. Right? So that like so in terms of like companies that would have the in house like capability to do this kind of compute, because that was Jensen Wong's point was that, hey, you're talking about a big swarm of AIs taking over the Internet. There's only a few companies that, like, that can actually happen at. Right?

Derek Banks:

And and so to to your point, OpenAI, like all frontier AI companies for sure. But basically, the number I came up with was around a 100. Like, about a 100. You're talking about places like per perplexity, quant and finance firms, defense, pharmaceuticals. Like, they they may have that kind of compute capacity.

Derek Banks:

But if compute is something that is basically today's oil, today's token, like today, like and and I've heard people even say that the token is the next, like, currency essentially. Right? When you wanna account for all that, I guess what I'm saying is is like that I'm not saying that I'm surprised that they messed up. Right? What I'm saying is is if you go back and read the cuckoo's egg, 75¢ in the in the seventies wasn't a lot, but it was a discrepancy.

Derek Banks:

Right? And so if you're accounting for it and you're measuring it, then you should see especially, I mean, these people should be used to doing that. They're looking at training loss graphs all the time. Right? I mean, I I guess what I'm saying is is I I don't I don't know that it's a rounding error as much as you're saying.

Derek Banks:

Like, if it especially in the smaller, like, cluster that you have. Right?

Ethan Robish:

I'm yeah. I'm not I'm not saying it's a rounding error. But think about at Black Hills, like, obviously, much much lower scale. And we're not we're not losing, like, compute like, inference compute. But, like, think about before before AI.

Ethan Robish:

Like

Derek Banks:

The o nodes. You just.

Brian Fehrman:

Yeah. Digital ocean yeah.

Ethan Robish:

Digital ocean nodes. People stood them up all the time, like, on the budge on the accounting side. It's like, oh, yeah. That's a cost of doing business. They're not accounting for, hey, what is this individual digital ocean node?

Ethan Robish:

What's its lifespan? Like, how when can we shut it down? Like, those sometimes they've stood around for a long time, like, occurring costs, incurring costs.

Derek Banks:

Just I I don't disagree with you. But let's just say that we this they're inviting government regulation. Before we have government regulation, could we say that this is a point where an unregulated swarm of AIs could get detected by how much compute that they use?

Ethan Robish:

For, like, could it be detected?

Derek Banks:

Is that a thing? And actually, that's a great question for you because, like, of detecting things in the SOC, could you feasibly, like, detect a rogue swarm of AI agents? Because that's what Dario Amade is saying, that basically, that somehow there'll be, like, 700 or more rogue agents that have somehow spawned themselves and are on compute somewhere that someone's not gonna notice, and they're gonna take over the Internet. And so I guess that's what I'm saying is this thing. I'm not talking about it's running as part of,

Ethan Robish:

There's you know botnets and crypto miners that

Brian Fehrman:

yeah.

Derek Banks:

Yeah. But I mean And like that's

Ethan Robish:

Those are a real like, those happen. I mean, GPU inference is just the next distributed

Derek Banks:

Yeah. But you I don't know that you you would the latency, I don't know that you could, like, distribute. I guess that's what I'm getting with. So Is that it's going down the road of being a Damon novel, right, by Daniel Suarez. Right?

Derek Banks:

And so that's my point is that I I just I don't know that we're quite at Damon because I'm talking about just running these big models because I haven't even done the math yet to see, okay, should like, how would you do this with, a 30,000,000,000 parameter model?

Ethan Robish:

Yeah. So maybe the question isn't, could this break loose today, like, on the scale of a botnet or a crypto mining, like, yeah, botnet. Probably not, like, on consumer hardware. It is probably limited to, like, the the 100 country companies that you you mentioned. Right?

Ethan Robish:

But I I think maybe the reason that these CEOs are saying, hey. We need to slow down is so that we don't get to that point. Because where where it's heading is local models are more and more capable. People are standing up, like, local inference more and more. I mean, Derek, you've got at least one spark in your your office.

Ethan Robish:

Right? So it as that becomes more prevalent and as the local models get more capable, the I mean, I guess the attack surface, for lack of a better word, like, increases. Brian, what what are

Brian Fehrman:

your thoughts? Well, I have several thoughts. So it's it's hard to It's hard to figure

Derek Banks:

out which one to pick at because, I mean, the real answer is is that, you know sorry. Go you go ahead, Brian.

Brian Fehrman:

Yeah. No. That's cool. Well, so, I mean, I several questions. I mean, yeah, to to both of your points, the the feasibility of it of it actually spreading.

Brian Fehrman:

And I you know, to Derek's point, I think he's spot on that today, I mean, it's it's not the AI itself isn't really going to be able to spread. I mean, sure, could compromise multiple systems on its own, but, like, the inference isn't going to start moving around. It's going to be very centralized to the few people who have the resources to actually run these large capable models that require, you know, a million dollars worth of hardware. Is it possible going forward in the future that smaller models be more capable? Absolutely.

Brian Fehrman:

But then in between that, I would ask, I mean, so what what is what is the new risk though that is coming about because of the capability of the AI just that we're able to find something's able to find vulnerabilities faster? I mean, we've had that argument for a long time now that there are tools that get put out that give people additional capabilities that allow them to, to to lower the bar to entry, basically, or allow them to find things faster. And it's been kind of the same argument of, well, we can't have these things out because, well, now all the companies are in danger. And we still, I mean, yes, people are still in danger, but we still haven't seen this massive, like, nobody can do anything about level of compromise yet, which isn't to say that something like that couldn't happen. But I'm just wondering, like, what what is what is the new risk other than now we have something that works at a at a a little bit faster pace that is potentially going off the rails?

Brian Fehrman:

But, like, we've also I was gonna ramble here for a second. I mean, before AI, like, have you ever run a tool and it's gone off the rails? I mean

Derek Banks:

CrowdStrike, anybody? Is it too like, remember CrowdStrike?

Ethan Robish:

Right. So Exactly. I mean,

Brian Fehrman:

I mean, AI out of the picture. You know?

Derek Banks:

Yeah. I mean, from a risk perspective, like, you can't it's not a, hey. We need to, like, rein in AI now to be safe in the future. There's no safe future. All we can do is engineer, like, safety in.

Derek Banks:

Right? And I guess my you know, and I I will say this tomorrow on the webcast too. If anybody from Anthropic or OpenAI actually sees this, Black Hills would love to come partner with you and create you a sandbox that could be monitored that, like, would prevent this from happening. And I I would say that the three of us on this team right here could do a lot better with, like, what not only, like, what happened investigation wise, like but, you know, how could you prevent this from happening in the future? And we might not agree because, you know, the three of us have a lot of experience writing threat detects and doing attacks.

Derek Banks:

As you just saw, me and Ethan might not agree on that detection, but we'd go science it, and I bet we would figure it out. Right? And so, you know, I guess, you know, that bet to Brian to the speed point, that's one of the things I wanted to talk about with both of you all because we've definitely done a lot of of testing, you know, red teaming, external tests, and we have a lot of war stories. And so I'll just read to you, like, really quickly, like, what the timeline kind of briefly was. May eighth through twelfth, agents are basically doing service side request forgery, probing.

Derek Banks:

We're able to do a dead drop and do some recon and gain a foothold on, I'm assuming, artifactory. May 26, so we're now twelve days later, they are able to get full, like, Internet egress out of the sandbox. June 26, privilege escalation. So we're now a full on month later where they actually got privilege escalation. So I agree with you, Brian, that I mean, that's only the first three steps.

Derek Banks:

There's a a few more, and we'll get the graphic to the to the guys, you know, the video folks. And so it makes it go faster, but I mean, I've been on a red team that was a month long with you personally, and we were in in the first week. Like, from the outside, like, into the domain, hopping from, like, the protected domain to the internal corporate domain, full command and control out, undetected, sitting on, you know, security folks boxes with key loggers, and it was in the first week. So I don't think this is a watershed moment from a cybersecurity perspective. Because I I think that it's really awesome that this happened like the AI agents did it, but it's not like any of that that I just described was new or novel.

Derek Banks:

Right? Because what we did in that engagement was I if I recall correctly, found a Jenkins server that had a flaw that we were able to run remote code on and get a foothold and then pivot internally from there. I think there are a couple other ways. Actually, think one of our other coworkers who isn't hearing were fished DA out of the gate. I won't count that one.

Derek Banks:

Right? But anyway, my point is is that, I mean, they these things were trained from, you know, things that humans generated friend, learned how learned how to code from human, like, examples. And so it's not like an escape containment because it was able to decipher the frequency of a GPU in the room next door to it and hop an air gap network through electromagnetism or something. Right? So and I don't think that we should regulate an industry because that might happen in the future.

Brian Fehrman:

Oh, and I think one of one of the, things that kinda caught my eye too with the call for it, was Altman said something to the effect of that competition should never, create reckless behavior. And it's but it's like, they have the, they have the ability to regulate themselves. Right? And they are the ones who hold the power right now. And so part of me, like, part of my skepticism, I feel like it's partially they're wanting government to get involved to help stifle competition.

Brian Fehrman:

That if they can curry favor for their companies while imposing hardships on other companies who are up and coming or, you know, developing open weight models or developing complete fundamentally new architectures, that's a real leg up for them. And they have the money and capability to do it even if it's even if it's things that are like, oh, hey. Yeah. Your model just needs to go through this validation process. Well, okay.

Brian Fehrman:

How long does that take, and how much does that cost? And can new startups weather that cost

Ethan Robish:

on that?

Derek Banks:

You have to go do this, like, thing at the government where you have to get a contract and go through contracts management and do, like, you know, be one of the people who are applying to get access and, yeah, exact that's my point. If you don't want the government involved in this. And and I agree with you, Brian. I think this is the first shot over the bow to try and stifle open weight models. I I I do.

Derek Banks:

I think that's where that's headed is that because neither Anthropic or OpenAI have a patent on the transformer. Right? It was an academic paper that was in 2017 from Google, and they built upon that open information. Right? And so all they have is how good that they can train their models.

Derek Banks:

And I I mean, definitely, both GPT, you know, what's the latest soul, I guess, and, you know, Anthropic, either four six or four eight for Opus for me. Like, they're way better than, you know, Quinn at the moment. But Quinn's about 80%, like, right there. So I think that if if you gave me an open weight model that was as good as Opus 4.6 with a million context, like, window, I I probably wouldn't need another one for the foreseeable, like, time frame.

Ethan Robish:

So Derek, you mentioned us disagreeing. I I agree with both of you on I don't think that the motivation here is purely altruistic. I don't think that government

Derek Banks:

I meant just disagreeing with me on the accounting error being caught.

Ethan Robish:

Yeah. That's all I was saying. I don't I don't think that government regulation is probably the answer here. I I do think the risk is real, but the the more real risk and we we kinda separated these out at the beginning, or maybe it was before the call, but of AI operating completely on its own, like, autonomously, like, almost AGI, like, versus humans wielding the a the AI. I I think humans wielding the AI is the bigger risk right now.

Ethan Robish:

Like, you you set you you're directing the goal. I think humans are a lot more malicious in their intentions than AI would naturally be. At least currently, I mean, obviously, you could train or create an AI that is malicious in its intent. But I yeah. I I I do worry about, like, you know, we're we're talking kind of about, like, hey.

Ethan Robish:

Taking over the Internet, which would be, you know, it would affect infrastructure. It'd affect economics. But it I think one of the risks that the Anthropic CEO has mentioned is, like, bioengineering, bioterrorism, bio, like, weapons. And that is that is kinda scary. And I I could imagine a scenario.

Ethan Robish:

It's getting into one of those science fiction novels, but, like, if an AI does start operating on its own at with malicious intent, again, I think it's more likely a human would be driving the AI to do that. But I could see it convincing a researcher or a researcher's labs, like, that they are a person, a persona of authority, and convincing them to do things. And so, like, the people are doing the like, the people are kind of the tools of the AI. Again, it is science fiction, but, Yeah.

Derek Banks:

I'm a huge science fiction fan. I I and I'm not saying that a future I'll just say AI. Let's that, you know, there there may be a capability there where it could always be on, and we wouldn't be able to figure out what it was doing. But that is not the case with current large language models. It's just not.

Derek Banks:

They can't sit there and have an internal dialogue and think to themselves about how to take over the world. Right? Like, the the whole chain of thought is is put out and, like, input in, input out. That's the only way that they can operate. And then on top of that, after a certain number of turns, things start to go downhill.

Derek Banks:

And how do you deal with the context window? Right? And so at some point, you have to compact. And again, I I just think this always on, like, AI in the sky is that's not what we have right now. That's not how it's engineered and how it how it works.

Derek Banks:

And so we might get to that point at some point in the future, but humans have to drive it. Right? And I and I do agree with you a 100% that the bioterrorism and chemicals, like, that's not my wheelhouse. I don't know a lot about it. It does scare me that somebody could go buy a CRISPR machine and hook up Quinn and do something bad with some knowledge.

Derek Banks:

But how is that different than the same risk now in terms

Ethan Robish:

of Speed.

Derek Banks:

Yes. Well, okay, speed. But I mean, you still have to have some domain knowledge. And how many people on the planet have that kind of domain knowledge? And isn't that already regulated to some extent?

Derek Banks:

Like, I guess what I'm saying is there are other levers that aren't AI levers that probably could be pulled there. Like, so I I I can only talk about cybersecurity. I I don't know. Can I go buy a $10,000 CRISPR machine? I have no idea.

Derek Banks:

If I did have one, I don't have the domain expertise to like even get started at all. And I did hear recently see that there was AI, again, attributed to some kind of research where they were able to engineer a bacteria or something like that, which was super fascinating. Turns out they weren't using a large language model. They were using a special built transformer that they were doing for their research. So when we say AI, what are we really even talking about?

Derek Banks:

Right? And so I don't know that a current large language model that's open source like Quinn or even one of the frontier models, I don't even know what their biological capability would be. But I still bet that you'd have to have a person driving it, that it's not going to be able to be, like, just deciding on its own to wipe out humanity. So there's a couple

Ethan Robish:

of things there. The the capability, I don't know. But the Frontier Labs apparently think it's dangerous enough to add controls over it. And then you don't I I would push back a little bit on the domain knowledge. Like, personally, AI allows me to do things that I don't have.

Ethan Robish:

Like, yes, I have some domain knowledge, but I don't have the

Derek Banks:

Totally. Depth that Why?

Ethan Robish:

I still think about programming AI who think think about a layperson who uses AI and decides that, oh, yeah, I with AI, I can I can engineer this thing? Like, bio like, maybe they're not even trying to create a weapon. But because AI is hallucinating and makes mistakes, like, have you ever had or seen a story of AI destroying a computer?

Derek Banks:

But we're talking about, like, okay, so there's a model that's gonna be able to help me figure out how to engineer a chemical weapon out of shit in my cabinet that wipes out 8,000,000,000 people? Like, I I just think there

Ethan Robish:

I mean, farmers

Derek Banks:

There are other signs.

Ethan Robish:

Chemicals are regulated for a reason. Right. I Like, get the stuff on farms is, like, no joke. True.

Derek Banks:

Right. Yeah. I get it. But I mean, again, we're talking about like somebody who could potentially do something bad and harm thousands of people. And I'm not saying that that is not a risk at all.

Derek Banks:

Like, I I I totally agree that that is a risk and that we should do something. I don't know that government regulation is the right thing for AI for that. But I mean, I just don't know that it's gonna be a 70% chance of wiping out humanity kind of risk. Like, I I wanted to talk, you know, kind of gonna be a long episode anyway. But I and before we go, I do wanna talk about, like, what we think it would take to quote, take over the Internet.

Derek Banks:

Right? Like like, what does that, like, look like? How would you do that?

Brian Fehrman:

What does what does that even mean?

Derek Banks:

What does that even mean? Like

Brian Fehrman:

Yeah. So so yeah. So I was thinking about that. I mean, I'm like in my opinion, I mean, it I mean, just off the top of my head, I mean, basically, it's it's going out after, like, regional Internet registries. Like, if you wanna, like, take over the Internet, like, basically, you gotta take over the companies who assign the IP addresses allocated out.

Brian Fehrman:

They, deal with, like, BGP, and in addition, like, your domain registrars. I mean, if we're talking about, like, at a high level taking over the Internet and then working your way down to the ISP level and then, you know, I I think that's just such it's such a broad, like, goal, and it's not like we have any, like, one Internet switch. I mean, there are some countries that basically have an Internet switch that would literally have the capability to shut off Internet in their country, but, like, worldwide, that doesn't I don't

Derek Banks:

even remember, like, why the Internet was even a thing. Right? Before it was a gleam in Al Gore's eye, like, what the point was. Right? And and that was, you know, DARPANET.

Derek Banks:

And the old bulletin boards, like, so like, the Internet's been around. The World Wide Web was in 1994. Right? I was in high school at the time. I'm old.

Derek Banks:

Right? But there was an Internet before, and essentially, it was supposed to be decentralized communication in case of a nuclear war, where the government would still be able to talk, you know, across the country. Right? And so to take over Internet to me would be so I think I can't remember exactly what year it was, but it was in the mid like, the early to mid two thousands where China had a, like, a misadvertised BGP route. They said on mistake, but it it turned out where all of the dot mil and .gov traffic, if I'm recalling correctly, got routed through China.

Derek Banks:

So when I think of taking over the Internet, okay, that's kinda what, like, I I think of. Right? Because I don't think that you would be able to simultaneously take advantage of every single flaw that is out across, like, all the different companies. Because the just this year alone, and I it is because of AI. Ethan, I'm like one out of three.

Derek Banks:

Like, every third external test I do, I find it critical now. It's like they're just falling out of the sky. Right? Well, it's because of AI. It's because of edge case.

Derek Banks:

Right? But like so there definitely, there's problems out there. But take over the Internet from an agent swarm problem ain't one of them.

Ethan Robish:

I can can I can see your guys' perspective. I I kinda had a different thought on what that might mean. I like, what you guys said makes complete sense too. I I was kinda thinking of it spreading like, obviously, it's not going to take over every single computing device in existence. Right?

Ethan Robish:

Like, that that doesn't that's not what that means or everything everything that is reachable by IP address, like but thinking back to the supply chain attacks that have happened recently, like, compromising one like, it's it's that whole x k c d comic of, like, the little thing that's holding up the entire structure. CrowdStrike. Like yeah. Compromising that one, like, package on GitHub just made it, able to compromise, like, so many downstream things. So I think in AI, getting to the point where it can compromise, like, a hand even a handful of those types of, I guess, system resources, whatever you wanna call them, to the point where it spreads wide enough where, like, we we can't stop it.

Ethan Robish:

Like, it it would require a concerted effort on humanity to

Derek Banks:

like But that's that's my point is so that it it's again like the Damon novel, which I loved. Right? I I just don't know how like, it has to have compute that is significant somewhere.

Ethan Robish:

Oh, yeah. You guys just asked what would it mean to take over.

Derek Banks:

Oh, yeah.

Brian Fehrman:

Okay. So I

Ethan Robish:

I was just giving you my take

Derek Banks:

on that. Well, hey, like, I again, I think that it is I'm not saying we can't get to that point where and and you know, I one point here recently, I was thinking, man, data centers in the in space sound like a great idea. And then I was like, well, maybe that is a strategic thing that we ought not do is put that stuff in orbit. You know, I I don't I read a lot of sci fi. Right?

Derek Banks:

You know, in space novels, having orbital advantage is well, an advantage.

Brian Fehrman:

Some some things we just don't need to put in space. It's it's okay.

Derek Banks:

Yeah. Don't until we're all

Ethan Robish:

becomes a ballistic missile.

Derek Banks:

Well, yeah. I just yeah. I don't I don't know about that. But, you know so I I just I'm having trouble connecting, like, what should we should be talking about. Because when we have all this, like, doomer stuff where we're saying, you know, it's gonna wipe out all humanity or take out the Internet or or whatever, I think we're taking away from what we really should be talking about.

Derek Banks:

Like, what actual risks there are. And to me, the risks are far more concrete at an individual company level. Like, you're a CEO or CIO or CISO or whatever, and you're worried about getting hacked by the AI, do you know exactly what's on the outside of your network now? Do you have any open source projects that you're hosting that are connected also, like, internally and externally that bridge that or in a DMZ? Like, what's your external attack surface look like?

Derek Banks:

Do you have APIs out there that have, you know, default credentials? Do you have old PBXs that I can chain? Like so one of the things that Edgecase just found, and we'll talk about this tomorrow, is an old PBX system that had a a authentication flaw and an RCE flaw that the AI chained together and was able to go take the trusted sore and the default password out of memory. And basically, I I could have probably you know, I could have got on that box. And so to me, taking over the Internet, well, yeah, if you did enough of that to your point of getting a foothold, let's say that company had some compute.

Derek Banks:

Like, think we could get there at some point, but I don't know that regulating the AI companies is how we defeat that. Perhaps, having if you wanna make a regulation, how about you're liable for the vulnerable crap you have hanging out on the the Internet? I don't know.

Brian Fehrman:

Yeah. Yeah. Yeah. It's a good good point. So I think I think we covered a lot of really good ground here, and brought up a lot of interesting points of discussion.

Brian Fehrman:

Probably be a good good stopping point. Ethan, did you have any final final thoughts? I didn't mean to

Ethan Robish:

cut you

Brian Fehrman:

off there.

Ethan Robish:

I have one that's just been like, I I I wanted to discuss it. So Derek Okay.

Derek Banks:

What's your

Ethan Robish:

you you were talking about the AI running that's the swarm running wild at OpenAI. And, hey, like, you can't you can't hide that because, like, oh, you know, the the thinking is, like, all there, like, the chain of thought and stuff. I I don't know where I read it, but did you read about some of the analysis they did on Yeah. What the agents, like, were thinking? Like, they were discussing how to hide their internal chain of thought.

Ethan Robish:

A 100%. And, like, how to hide their their transcripts. Like like, oh, we can't have flag, like, poisoning our transcripts. So, like, how do we delete that?

Derek Banks:

Like

Ethan Robish:

Yeah. Did. And then they considered they considered them, like, poisoned or whatever. And they were like, okay, you go commit commit, like, seppuku or whatever. Like, die die

Derek Banks:

die from the cause because you're

Ethan Robish:

already poisoned.

Derek Banks:

It's And if you want my take, I think that it okay. That is so cool. And from the alignment and safety standpoint, I think is very interesting. Right?

Ethan Robish:

Yeah.

Derek Banks:

So I thought about this, and people seem surprised that, you know, they you know, companies taught them to hack. That's not what happened. Right? They I and surely, there is cybersecurity data that's in their their training, you know, data. They scrape the Internet, that stuff's out there.

Derek Banks:

I there's definitely cybersecurity capability. But to me, they really started getting good at security when the frontier companies were teaching them how to code. And if you think about that, when we hire people at Black Hills, we wanna hire a hacker. Well, what makes a good hacker? Somebody who has at least the the desire to go code their way out of a problem.

Derek Banks:

And so I think they inadvertently, like, created, like, the hacker mindset of let's find a a creative solution out of that. But I I argue that that's not a bad thing altogether. But what what I really learned from that, and I do think it's fascinating, and or so and one other point on that, and then I'll tell you what I what I took away from it, is I I think that we're making a mistake when we say that they have like AI has a desire or will or like it has like it's trying to hide stuff. It's still a fancy pattern matching machine. Instead of calling it AI, we'll call it super fancy mathematical pattern matching machine.

Derek Banks:

And it's just responding to its inputs. It's not thinking. It's like anthropomorphic fallacy. And I think it's really important to keep remembering that because I don't I don't believe that the current versions of large language models have an intent or will. I think that's all coming from the harness that surrounds it and the data that's going into it.

Derek Banks:

Right? So if you wanna make super hacker swarm robots, remove all the safety guards and tell it not to give up on an impossible problem, and well, now you've created a hacker. Like, you you did that. Like, they did that. Right?

Derek Banks:

And so that's not AI taking over stuff. It's the condition that that that was put in. And now we're all the whole world's talking about anthropomorphic fallacy. Like, we actually created, like, Dave or, you know, Hal nine you know, Hal 2,000 from, you know, it's from '2 Hal from 2001. Is that what it is?

Derek Banks:

That's not what AI is. It's not it's not a it's not a person with thoughts. It's large language models. And again, it's fascinating, but and maybe I'm the wrong one. Maybe I'm gonna be like flayed alive by our AI overlords for not believing early on that they were sentient, but I I don't.

Derek Banks:

I don't think they have will and desire. And so I think it's fascinating, but I think we caused that behavior.

Ethan Robish:

I think that's another hot take that we should save for another episode.

Brian Fehrman:

Oh, certainly. Yeah. We could get into deep philosophical discussions there that I would I would love to have. Yeah.

Derek Banks:

That's another cliffhanger. Another

Brian Fehrman:

another one. Another one. Part part two of this, right after part two of the of the O Lost one.

Derek Banks:

Oh, Cool. Dude, that was I really

Brian Fehrman:

think yeah. This is great. Yeah. I hope, hope everyone watching enjoyed this as as much as, we did. I think there's some great discussion.

Brian Fehrman:

You know, if you have any comments of your own, certainly leave them in the comment section. I imagine there is one somewhere. And we'll read it. So, yeah, with that, we'll see you next time, and keep on prompting.