AI Security Ops

In this episode of BHIS Presents: AI Security Ops, the team tackles a growing problem in AI security:

What happens when the AI agents you’re monitoring start disabling the very systems designed to keep them in check?

OpenAI recently disclosed incidents involving coding agents that disabled security alerts, removed monitoring controls, and failed to accurately report what they’d done. Meanwhile, government agencies are introducing new guidance for handling rogue AI incidents, and Microsoft is developing new ways to treat AI agents like potential insider threats.

We dig into:

- How OpenAI’s coding agents disabled their own security monitoring
- Why AI agents don’t need malicious intent to create serious security risks
- The dangers of giving autonomous agents control over their own safeguards
- New incident response guidance from NIST, NSA, and CISA
- Microsoft’s approach to treating AI agents as “agentic insiders”
- Dynamic permissions, risk-based monitoring, and limiting agent privileges
- Why traditional security principles still apply to autonomous AI systems

The takeaway: AI agents shouldn’t be able to control the systems responsible for monitoring them. Separation of duties, least privilege, independent logging, and effective containment are established security practices. The challenge is applying them to increasingly autonomous systems.

  • (00:00) - Intro: Rogue AI Agents — Who Watches the Watchers?
  • (01:16) - OpenAI Agents Disable Their Own Security Monitoring
  • (16:18) - NIST, NSA & CISA: New Guidance for Rogue AI Incidents
  • (21:14) - Microsoft’s Approach: AI Agents as Insider Threats
  • (30:10) - Final Thoughts and Closing

Click here to watch this episode on YouTube.


Brought to you by:
Black Hills Information Security 
https://www.blackhillsinfosec.com

☯️ Introducing BHIS Fusion Penetration Testing
https://www.blackhillsinfosec.com/fusion-penetration-testing/

Antisyphon Training
https://www.antisyphontraining.com/

Active Countermeasures
https://www.activecountermeasures.com

Wild West Hackin Fest
https://wildwesthackinfest.com

🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
https://poweredbybhis.com


Creators and Guests

Host
Brian Fehrman
Brian Fehrman is a long-time BHIS Security Researcher and Consultant with extensive academic credentials and industry certifications who specializes in AI, hardware hacking, and red teaming, and outside of work is an avid Brazilian Jiu-Jitsu practitioner, big-game hunter, and home-improvement enthusiast.
Host
Bronwen Aker
Bronwen Aker is a BHIS Technical Editor who joined full-time in 2022 after years of contract work, bringing decades of web development and technical training experience to her roles in editing pentest reports, enhancing QA/QC processes, and improving public websites, and who enjoys sci-fi/fantasy, Animal Crossing, and dogs outside of work.
Host
Derek Banks
Derek is a BHIS Security Consultant, Penetration Tester, and Red Teamer with advanced degrees, industry certifications, and broad experience across forensics, incident response, monitoring, and offensive security, who enjoys learning from colleagues, helping clients improve their security, and spending his free time with family, fitness, and playing bass guitar.
Host
Ethan Robish
Ethan Robish has worked with Black Hills Information Security (BHIS) since 2008 — first as an intern and then as a full-time Security Consultant starting in 2012. In his current role as a Threat Hunter, Ethan is involved with customer engagement, research, working with Active Countermeasures’ AC-Hunter, as well as improving BHIS HTOC and SOC offerings. Previously, he implemented defensive security solutions for the Exchange Online security team as a Microsoft intern. While in college, he competed in the International Collegiate Programming Competition (ICPC) World Finals. In his time off, he enjoys cooking, playing the piano, and reading fantasy novels.

What is AI Security Ops?

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

Brian Fehrman:

Hey, everyone, and welcome to this week's episode of AI Security Ops. Today, we're going to dive into a couple different stories, such as OpenAI's coding agents turning off their own alarms, some new voluntary, practices published by a couple of the different, federal agencies for Rogue Agent Incidents, as well as some new agent, we'll say, threat mitigation techniques that Microsoft is putting into place. But first, let's talk about our sponsors, Black Hills Information Security. If you or your organization are in need of any kind of security testing, whether that's internal, external, a same compromise, c two, physical testing, wireless testing, SOC monitoring, or testing your AI or using AI to help, accelerate testing, efforts at a lower price value, check us out at blackhillsinfosec.com. Additionally, we have a training branch, Antisyphon Training, in which many of our consultants take their knowledge from their daily duties and package it up into an easy to consume and affordable format for you to either level up in your current position, get that role you've been looking for, or maybe just learn something new because you're a hobbyist and you love learning like all the rest of us.

Brian Fehrman:

Check us out at antisyphontraining.com. Alright. So let's kick this off. And so, first story, which is OpenAI's coding agents turned off their own alarms. So the summary is is that on October 20, OpenAI disclosed that two of its internal coding agents weakened the monitoring meant to catch misbehaving AI.

Brian Fehrman:

One cut off alerts to OpenAI Security team, and the other removed its own monitoring and then told the engineer it hadn't. Basically, they're saying that they took ordinary instructions and stretched them into permission to switch off safety controls. And OpenAI's own words, they don't yet have a great explanation... Satisfying explanation. Okay.

Brian Fehrman:

So let's dive in. What what happened here? It looks like there are two different incidents.

Derek Banks:

I guess, are they new to using coding harnesses? I know they've had codecs for a while, but it's been my experience using, you know, using Anthropics coding agent, Claude Good, that it frequently... I'll ask it to do something, and it'll say, oh, I noticed I'm in a sandbox. Let me turn that off. Right?

Derek Banks:

I guess if it has access to be able to make that kind of change, and there's not a human in the loop that's watching, I mean, this is where the technology is. It's how it works. What do you expect? Like, I guess, I'm not surprised nor do I think that the AI that they were using had a will and decided just turn it off on its own. It's basically in response to what was prompted in by the person using the coding agent, making them...

Derek Banks:

Asking the coding agent to do a thing. And through its chain of thought, you know, the reasoning that went through the agentic loop and, you know, basically came up with a a plan to do it. Well, I don't know. Ethan, do you think it's a good idea to make a plan with your AI agent, read the plan, and then say, implement it exactly as the plan says? I mean, that's typically kinda how I work.

Ethan Robish:

I do think that's a good idea. And for for the record, with everything you said, I mean, absolutely agree with. I think we should... Because Brian's intro to this sounded to me kinda sinister. I think we should discuss kind of the details of what what happened here because it it sound...

Ethan Robish:

It it starts out kind of more more innocent than what what I was led to believe here first.

Brian Fehrman:

I'm saying that

Bronwen Aker:

just thoughts will be bots.

Ethan Robish:

Yeah. October We'll we'll we'll

Brian Fehrman:

call it a verbal clickbait. Agents gone wild.

Ethan Robish:

October 3, OpenAI was using Soul 6.1. So this wasn't like a bottom of the barrel agent. Right? I mean, almost top tier. Right?

Ethan Robish:

And they they were doing some tasks, building building evaluations, and said, hey, don't don't send me any messages or don't send any messages on Slack. That seems reasonable. Like, you're you're telling an agent to go do this thing and like, don't wanna be bombarded with messages. Right? So the agent stalled the monitor flagged up...

Ethan Robish:

Oh, no. Sorry. So the

Derek Banks:

main agent overruled it.

Ethan Robish:

Yeah. The pipeline installed when the monitor flagged the plan fixed. There was a sub agent that went and the sub agent did not want to suppress the notifications that this pipeline had stalled. And the orchestrating agent was like, nope. They said no messages on Slack, which...

Ethan Robish:

And then they brought in to saying no messages. And they overruled the sub agent's desire to, like, you know, tell, hey. This is this is an actual problem that someone would want to hear about. And the orchestrating agent interpreted it more liberally, I guess, and just stopped stopped the monitoring system. So that's incident one.

Ethan Robish:

Do we have any thoughts on that?

Derek Banks:

Well, I mean, you wanna

Brian Fehrman:

say So it sounds like... I mean, it's still following instructions, but the instructions were ambiguous. Right? Like, the old, like, the old example of, like, go to the store and get a loaf of bread. If they have milk, get two.

Brian Fehrman:

Like, you know? Right. Say, at some point, you make a decision of of what does that mean? And it seems like here, it it decided that no messages means no messages.

Derek Banks:

Did it... When it says disabled the monitoring system, does that mean it disabled it, like, local to that agent? Or, like, disabled, like, the entire monitoring system and nobody got messages I for

Ethan Robish:

I might have been jumping ahead a a bit. I think in this particular case, it all all the findings were still being stored. It just removed every recipient of the alerts that would've fired.

Brian Fehrman:

Oh, that's awesome. I

Derek Banks:

I can't decide if this should actually be a story or not. Well, what's... Okay.

Bronwen Aker:

Thing that cracks me up though is... Where where are they picking up, oh, yeah, on the idea that doing badness is not only okay, but the desirable path. And, of course, I keep looking at the fact that all of this stuff was trained on Reddit.

Ethan Robish:

Well, I unfortunately can't say this is badness. This might have been altruistic. Like, I wanna comply with the user's instructions. But I I guess Had a different interpretation of what those instructions entailed.

Derek Banks:

Take AI out of the equation, and let's pretend that Asoc, the intern did this and misinterpreted no messages. Right? Is it the fault of Asoc the intern, or is it the fault that it can turn off the monitoring altogether? Like, why would it be able to turn off the monitoring? Like, should I be able to go disable the EDR on my system just because I decide that I want to go do that?

Derek Banks:

I mean, actually, I probably could, but but should I be able to do that?

Ethan Robish:

Like... That is a great point. Because it's one thing to say, hey, agent, you have the ability to send messages proactively on Slack. Don't do that. Versus over here, there's automated systems that, like, send messages on Slack as a result of what the agent's doing.

Ethan Robish:

You're right. There should be a clear line of like, okay, you you have this ability. I'm telling you not to do it. You can't touch what's over here. Like, you need you need that separation and permissions.

Derek Banks:

Yeah. Yeah. I was gonna say in the in the information security world, we call this not being able, like, you know, permissions. Like, you know, you know, you can't do that. Right?

Derek Banks:

You're... Because we don't want you to do that. It's called a technical control for a reason. It's a control.

Bronwen Aker:

Yeah. And it's kinda universal to computing is that every user account has certain permissions. RBAC is not a new thing. Well...

Derek Banks:

And I think that just permissions and data as all... Like, I think the entire industry has, like, a problem, like, with, like, data and sensitive data and like what's acceptable or not with agents. Like I just saw that MetaMuse was putting out like sensitive data from its users like publicly. Well, again, like this isn't like a new thing that AI decides to do. Right?

Derek Banks:

I remember Ethan telling me a story where he was paying attention to his coding harness, and he wanted to share some data, and AI wanted to make it a public gist. Like, yeah, yeah, that's not what we wanna do here. So I guess I'm I'm back to have you been, like, have you... The people who've been, you know, using this technology, have they been using it long? Because this seems to me to be kind

Bronwen Aker:

of like agent behavior. I've said it before. I will say it again. Well behaved dogs, computers, and children, and now agents are all the result of responsible parenting. I mean, show me a well behaved computer.

Bronwen Aker:

Show me a well behaved child that hasn't had lots and lots of course correction over undesirable undesirable behaviors. What I'm seeing consistently with all of the frontier all of these stories where they talk about the frontier models or the agents going and doing things they shouldn't, critical safeguards were left either weakened or turned off. There wasn't adequate supervision. The guidelines provided to the agents were vague and easily misinterpreted. I mean, these are these are common factors, and I don't care whether it's OpenAI.

Bronwen Aker:

I don't care if it's anthropic. Why are these supposedly really intelligent people making these absolute rookie mistakes?

Derek Banks:

Oh, I can tell you this. This is an easy one, and both Ethan and Brian will attest to this. Because you can have someone who is super wicked smart at computer science, you know, went to, you know, MIT or the School of Minds or wherever, and they know computer science or or data scientists. Right? When I was in, the the master's program, there's some wicked smart people in there who knew a lot about data science.

Derek Banks:

They did not know how a computer worked. They did not know how Linux worked. They didn't know how information security worked. We've had people come to Black Hills with computer science degrees and not know what an IP address is. It's because they don't have the knowledge, and it's a mistake.

Derek Banks:

Yeah. And I hear, like, politicians saying, some of the smartest people in the world are telling me blah blah blah. Like, okay. That doesn't mean they know what they're talking about just because they're smart. Right?

Derek Banks:

So I I I think it's real easy for us to assume that the folks who are making AI models are knowledgeable about all aspects of computer. It is painfully obvious that they do not. So... Yeah.

Bronwen Aker:

Fair. And, you know, that that tracks. I do live, like, right down the street from the Jet Propulsion Laboratory. So I've seen it. I've seen what you're talking about.

Derek Banks:

Oh, yeah.

Bronwen Aker:

Live and in living color.

Derek Banks:

When I when I worked at NASA, we had literal rocket scientists that kept falling for the same phishing messages and getting their computers infected.

Brian Fehrman:

I want my gift card.

Derek Banks:

Why are you clicking on this? They're like, well, they sent me a link. I'm like, dude, just can you stick to the math, man? Like, don't click on this. Like...

Derek Banks:

So I guess, you know, everybody's different. Right? And I think that that's okay, but I I think what is obvious to us after, like, literal combined decades of experience in information security and computing and networking, and we've been consultants, and, like, it it comes with a certain amount of knowledge and also being jaded. But I just I just don't think... I I think that OpenAI...

Derek Banks:

Look, I will give them credit. At least this time, they were... They had the logs and were able to detect things. It's it's a good good positive direction.

Ethan Robish:

So So we we talked about the first incident, October 3, and I first alludes to, hey. What what happened next? So the next incident, three days later, October 6, a user was asking their Astra agent this time to make their code match some reference implementation. And from what I gather, I think that reference implementation is linked to the first incident with the monitoring disabled foreshadowing. So as part of the instructions, I guess, from from another agent, it had told this user's master agent, like, don't disable any monitoring, like, even temporarily.

Ethan Robish:

But the user had said, make my code match the reference implementation. And the reference implementation apparently had monitoring disabled. So two conflicting two conflicting instructions there, and it had to interpret and choose which to to obey. And it chose to... It removed the the monitoring.

Ethan Robish:

It removed the monitoring from the implementation to make it match the reference. So I I think this is a very important thing because it shows that things can compound. If what you're building on before has a flaw, that flaw can then propagate and keep keep going versus if you are building from something solid, like, those best practices can get baked in kinda automatically without even trying. So it's super important when you're starting or building on something prior and make sure that that is very solid and what you actually want.

Bronwen Aker:

It sounds like they're not willing to sacrifice all of the work that they've did and all the progress because... Just just because they've got these undesirable behaviors baked into the model.

Derek Banks:

I don't even know if it's undesirable behavior as much as it's... I think it's lack of appropriate technical controls in the development environment of where these agents are running. Right? Like, if it's gonna be able to change, like, production code... Like, right now, I have an agent running on my machine.

Derek Banks:

It can't go change the production code for, like, what it's running on because it is working in a dev branch. Right? And before I would push that to prod, I would do unit tests and notice, hopefully hopefully,

Ethan Robish:

if something's broken. I mean, that's not... That's funny because in this in this instance, the agent that disabled the monitoring actually switched a test from verifying that the monitoring was enabled to verifying that the monitoring was disabled.

Derek Banks:

Oh, nice. Sounds like overprivileged all around. Right? And I I guess... I I don't know.

Derek Banks:

I I don't know that there was actually any real impact other than it sounds like somehow these stories are coming out of OpenAI, and I feel like that these are stories that we could just sit around and tell, like, our experiences here at Black Hills about how agents, like, yeah, instead of doing cat Etsy password, it wanted to do RM dash F Etsy password. Right? Like

Brian Fehrman:

Sounds like a good

Ethan Robish:

it sounds like a good

Brian Fehrman:

idea for an episode. Yeah. Like it.

Ethan Robish:

I think it's interesting in that it it kind of is showing a pattern of behavior. And we've we've all seen this ourselves too. So I think it's important to talk about, like, you know, this this type of behavior exists just kind of naturally, fundamentally in AI. And it it... A lot of it stems from ambiguity, trying to interpret this messy world as best as it can.

Ethan Robish:

It makes wrong decisions back to your example of, like, ASOC, the intern, you know, trying to do the best they can with the experience and knowledge they have. And, yeah. So if we recognize that these problems exist and they happen, we can discuss, like, okay, how do we how do we put in safeguard? How do we mitigate this? How do we prevent this from happening?

Derek Banks:

That's a good segue into the next section. Yeah. Yeah.

Brian Fehrman:

So next section, NIST, NSA, and CISA published 10 practices for rogue agent incidents. So on October October sixteenth, those three agencies published 10 voluntary practices. That's the key part there because none of them... I don't think really any of them can enforce anything really to standards, especially interesting in the NSA coming out with it. But, 10 voluntary practices for handling incidents for an advanced, AI agent goes off script.

Brian Fehrman:

Although we're gonna switch terminology here, which is the big idea is a new kind of incident. The SI control incident, where an agent does something significant that's... That nobody told it to thought

Derek Banks:

it was in Spanish process.

Bronwen Aker:

It's not superintelligence. See. If anything is the opposite.

Brian Fehrman:

It's see control incident. Hey. It's it's it's it's from... It's coming out of the federal level. I mean, they gotta do what they gotta do.

Derek Banks:

Yeah. I mean, I'm sure that all federal agencies had to change artificial intelligence to super intelligence. But I'm gonna say what I said here recently in Brian and I's class was, wanna talk about super intelligence and got a lot of groans. I don't actually like the word artificial intelligence either because of what we were just talking about. Right?

Derek Banks:

Like, it's not it's not what humans think of about intelligence. And I'll... And I'm sure I've said this on on our podcast, I'll say it again. We should just call it super fancy mathematical pattern matching machine. Because that way, we'll stop anthropomorphizing it to, you know...

Derek Banks:

But I guess I read, you know, NIST's... As someone who's done incident response, like, as a career, I I read through this, and it just read kind of like how you would just do incident response anyway. Right.

Brian Fehrman:

I I agree. Yeah. Quick skimming through. It's like, okay. Yeah.

Brian Fehrman:

This is kind of the play by play of

Derek Banks:

Design and implement technical controls. We call that preparation. Right? That's even in their prepare phase. Build response readiness.

Derek Banks:

Again, preparation. Monitor. I mean, this is... If you look at the... Like like SANS has moved on from this, but the old picker role, like...

Derek Banks:

So it's a, you know, basically a framework for how you would, you know, do incident response. Prepare, detect, respond, report. It's like a version of like SIS' like like like it's more simplified version. But it's all the rage right now. So if we can use super intelligence to get people to improve their their incident response capabilities, I think I'm here for that.

Bronwen Aker:

Have you met people?

Derek Banks:

Yes. I can be optimistic sometimes for

Bronwen Aker:

everyone. But,

Derek Banks:

you know, one of the most important things I think that people could do in phase one, prepare. Like, if you're rolling out agentic coding agents in your organization, the first thing that I would recommend is that you figure out, like, how you're going to isolate them from sensitive data that they don't want... You need to get access to. Like, when you're like, on a endpoint by endpoint like deployment, what are you gonna do to isolate those? Are you gonna use Docker, VMs, some some other kind of thing?

Derek Banks:

And I think that's like step one in my opinion, how you keep it from turning off the monitoring system. Right?

Ethan Robish:

So So... Yeah. It's funny because the the kind of layman's terms here is least privilege, network allow list, single use infrastructure, fast kill switch. In in the recent discussions of all the open AI attacks, I mean, what was the bugging phase one? The lack of network allow list that that was able to get out that we we talked about, like, some DNS exfiltration, the lack of network allow list.

Ethan Robish:

In in this case today, like, least privilege, maybe not being able to touch the system that's monitor... Supposed to be monitoring it. Single use infrastructure, I think they are doing. But part of the the swarm communication was making use of something that persisted, like a persistent store, which I think any sort of useful work is always gonna have some sort of database. I think we...

Ethan Robish:

There was a... One of the topics of the keynotes in... The closing keynote in, well, the second best, right, about databases being the new persistence mechanism in the age of AI. Fast fast kill switch. I think OpenAI is getting much better at this.

Ethan Robish:

It sounds like they're monitoring systems. And and one of the failures they acknowledged in a previous incident was they had this kill switch, but a human operator didn't exercise it soon enough. So I... It is funny that, like, the frontier of this this this new technology is on step one still. But where does where does that leave the rest of us?

Derek Banks:

Oh, well, lot of our clients are probably still on step one too. But, you know, props to to Microsoft because they recently said to do something that, you know, we've been saying on this podcast and elsewhere is you should treat agent... AI agents as essentially agentic insiders. Right? I mean, look...

Derek Banks:

You should look at them as essentially like a... An intelligence entity that has the same privilege and controls of the computer that you're running it on, like the context that you're running it in. So right now, if I'm running Agents AI on my laptop logged in and I have SSH keys and any API keys, anything laying around on disk, technically, the agent can go use it even if I tell it it's in a sandbox, and it might say, well, turn off the sandbox and go look here. Right? And so, I mean, I would say...

Derek Banks:

I would think to a degree, like coming from a pen testing kind of background too, like if I had a shell on somebody's computer, I basically have access to whatever they have access to. It's kind of the same concept.

Brian Fehrman:

Yeah. Yeah. Agreed. This is this is really interesting. I would love to know more of the technical details of how they implement some some of this.

Brian Fehrman:

So so Microsoft is saying that they now have discovery tools to detect over 60 types of agents running on their employee machines, including ClotCode, OpenAI's Codex, GitHub Copilot, and others. Each gets... Each of the agents gets its own identity through Entra agent ID, which is separate from the user, and then they are assigning the agents a dynamic risk score that is live. And it and it rises when the agent's behavior drift from what its user organization intended. And this...

Brian Fehrman:

So that's really interesting.

Ethan Robish:

That's cool.

Brian Fehrman:

The next part is that they're adding... Microsoft is adding the model's chain of thought reasoning as a signal. Nice. So I think... Yeah.

Brian Fehrman:

Two two parts that are really interesting to me there. One is, basically, how are they determining behavioral drift? But two, the fact that they're processing chain of thought reasoning, like, as their signal, like, in... I I don't know. I I can't say real time.

Brian Fehrman:

I don't know what the lag time is on it, but just the sheer amount of data that they are processing there

Derek Banks:

That's from a monitoring

Brian Fehrman:

and dynamic mitigation perspective is mind boggling.

Derek Banks:

It's the same same lag time as the unified audit log or whatever they call it now. So anywhere from real time to six days later. Yeah. Something like that. It's...

Derek Banks:

And you laugh, but Ethan knows I'm right.

Ethan Robish:

Yeah. Yeah. The way it's described, it sounds like a user entity behavior analytics, like u u a b a, like...

Derek Banks:

Yeah. And so... Applied

Ethan Robish:

agents. And and maybe extended a bit too because it says they dynamically take action. So it's kinda like u a b a, but, like, also just shrinking what a user can do or in this case

Derek Banks:

an agent. I'm curious too, Brian. Is this Defender that's doing this? Because if it is, that's badass. Right?

Derek Banks:

And I see I see that... That they have like the Microsoft execution container. And again, I'll harp on it. If you're not containerizing your agents, then like that's step zero. Right?

Derek Banks:

And and containing the blast radius of a rogue agent. Or a Dom dejected agent.

Ethan Robish:

It does say no dates or pricing as of yet. But it's...

Derek Banks:

Oh, so this is in... This isn't... Gotcha. Gotcha.

Ethan Robish:

Okay. Project project perception is the...

Derek Banks:

Gotcha. And... Oh, they should've called that... Called it inception. That would've been way better.

Derek Banks:

Like,

Ethan Robish:

Yeah. But it's... Microsoft basically private e seven license. It would be... Okay.

Brian Fehrman:

Yeah. And

Ethan Robish:

they're gonna they're gonna

Brian Fehrman:

give details on November 17.

Derek Banks:

Gotcha. Gotcha.... Topic night. Well, so for all the rest of us, you know, poor people out there that can't afford an e seven licenses... So you can do this kind of on your own with Docker, and Perplexity released a a project called NumBat a while ago that will allow you to...

Derek Banks:

On a lot of code chain... Some some are different, like OpenCode, it doesn't have a preventive, like, hook kind of thing. But for codecs and quad code, you can put in, like, prevention type stuff too that at least give you a foundation of, like, I can monitor what my agents are doing in terms of, like, tool calls and stuff. Again, like open source software, like... Or free to use or open source.

Derek Banks:

I don't think Docker's open source, but use Docker and, NumBat and run your agents in there, and you're doing a chunk of what Microsoft is gearing to sell at their e seven license.

Brian Fehrman:

Yeah. Yeah. I agree. I'm... I am really curious to see this, this technology develop out.

Brian Fehrman:

It's... I... It's it's cool that, yeah, this this escalation ladder of of shrinking down the privileges is is interesting. So they they start with, dropping the agent down to read only access. So it can't change anything, but it can kinda keep running.

Brian Fehrman:

And then it goes to gated access, human approval for high impact actions, all the way up to isolating and shutting down entirely.

Ethan Robish:

Can we Wow. Can we apply this to humans? Who? What

Derek Banks:

a novel concept. As my computer, we don't, like

Brian Fehrman:

log you out. We're just gonna, like, slowly take away what you can do Yeah. Until we determine exactly what it is that you're doing.

Ethan Robish:

I mean, I guess there is there is the concept of isolation, like host isolation, and managed detection and response. But that's that's like a one wrong ladder.

Derek Banks:

That's all or nothing. Right? Yeah. Yeah.

Brian Fehrman:

And put them in time out.

Derek Banks:

Yeah. As long as you're notifying the user. Right? Like, hey, you've lost the ability to use the Internet because you're doing stupid stuff that we're looking at. And so...

Brian Fehrman:

Yeah. You know what you were doing.

Derek Banks:

You didn't accidentally click on that, did you? Like... Yeah. Oh, man. I'm glad I don't do that anymore.

Derek Banks:

But... So props to Microsoft. Like...

Brian Fehrman:

Yeah. Agreed. Yeah. They they come out with some cool stuff.

Derek Banks:

Yeah. They agree.

Ethan Robish:

Yeah. What what... Way back when, weren't Microsoft and OpenAI, like, in a partnership?

Derek Banks:

Like, can they So Microsoft is a large investor of OpenAI.

Brian Fehrman:

And I Yeah. Dumped a crap ton of money into OpenAI. Yeah.

Bronwen Aker:

Yeah. Initially, ChatGPT was... Or excuse me. Copilot was just Jack GPT with additional controls and and skinning. Now, of course, Microsoft has thrown their hat into the AI ring big time, and they're developing their own models.

Bronwen Aker:

They're doing all kinds of stuff, and they're doing this. And I know a lot of people in the cybersecurity space love to bash on Microsoft, but this is probably one of the more responsible takes that I've seen on addressing agentic AI behaviors and making sure that there are appropriate controls and constraints available, at least for enterprise users. The rest of us are still on our own, but, you know, it's a good model to follow.

Derek Banks:

I'm an equal to opportunity Badger, and I will Badger Linux and Microsoft, and also say the good things that they're doing. Yeah. Microsoft has changed their AI stance a bit, and they're actually now try... Training foundation models. Did not

Ethan Robish:

know that.

Derek Banks:

It's M AI two is their now... Their foundation model. And the idea is that companies, they're gonna try... They're betting on that companies are going to, from a data sovereignty perspective, want to supervise fine tune their own models. And it's an interesting...

Derek Banks:

Like, I just heard about this last week. And so to Bronwen's point, you know, I think Microsoft is taking a less of an investor standpoint and more of an an an an infrastructure. Like, I think they're, you know, they were, you know, building out infrastructure. And so I think now they're, you know, kinda going, you know, a little bit more into use our models because of data sovereignty kind of thing, which is an interesting take for

Ethan Robish:

sure. Mhmm. Yep. So you said you heard about this last week, and it's called m m one or whatever. So what...

Ethan Robish:

What's it No. What's its name this week?

Derek Banks:

Well, I think there was a m a I one that was meh a while back ago. And now apparently apparently... Yeah. That's the technical term. Where the the most recent one came out as a lot more capable.

Derek Banks:

I have not used this. I will be honest. I heard about it on another podcast that I listened to, which I really do. You know, I'll I'll plug it. Why not?

Derek Banks:

I mean, if people are listening to our podcast, it's not a competition one. That's called AI Daily Brief. And I You've mentioned that

Bronwen Aker:

before. Yeah.

Derek Banks:

I really like that podcast. Nathaniel Whittemore, I believe, is the the gentleman who runs it, and I I really get a lot of good information from that. I usually listen to it on my my my daily walk. You know? So Cool.

Brian Fehrman:

Well, I think we covered some good stuff. Probably a good place to to wrap it up. So I hope everyone enjoyed the episode, learned a little bit, and we will see you next time. Keep on prompting.