AI Security Ops

In this episode of BHIS Presents: AI Security Ops, the team tackles one of the most urgent — and misunderstood — problems in modern security:

How do you actually secure AI agents?

Not hypothetically. Not in theory. But in the real world — where agents have access to your filesystem, your credentials, your network… and are making decisions on their own.

The answer isn’t a single control or tool — it’s a maturity model.

From “YOLO agent with full access” to fully instrumented, controlled, and observable systems, this episode walks through a five-level maturity model for agentic security — and what it actually takes to move up each stage.

We dig into:
• Why agentic AI introduces a completely different security model
• What “Level 0” chaos looks like in real organizations
• The risks of giving agents unrestricted access to systems
• Why containment is the first real step toward security
• How sandboxing changes the risk equation
• The importance of logging, monitoring, and visibility
• Where most organizations are actually operating today
• Why skipping steps in maturity creates hidden risk
• How to think about blast radius in agent design
• What “fully enforced” agentic security actually looks like

This episode explores a critical shift in AI security: you’re not just securing models anymore — you’re securing autonomous systems.

⸻

📚 Key Concepts & Topics

Agentic Security
• AI agents with system-level access
• Autonomous decision-making and execution
• Expanding attack surface beyond prompts

Security Maturity Model
• Level 0 → Level 4 progression
• Incremental risk reduction strategies
• Why maturity matters more than tools

Containment & Sandboxing
• Limiting blast radius
• Isolating agent execution environments
• Preventing lateral movement

Monitoring & Observability
• Logging agent actions and decisions
• Detecting misuse or unexpected behavior
• Building visibility into autonomous systems

Defensive Strategy
• Designing for least privilege
• Avoiding “full access by default”
• Treating agents like untrusted users

#AISecurity #CyberSecurity #AIAgents #LLMSecurity #ArtificialIntelligence #InfoSec #BHIS #AppSec #AgenticAI
----------------------------------------------------------------------------------------------
About Brian Fehrman - https://www.blackhillsinfosec.com/team/brian-fehrman/
About Bronwen Aker - https://www.blackhillsinfosec.com/team/bronwen-aker/
About Derek Banks - https://www.blackhillsinfosec.com/team/derek-banks/
About Ethan Robish - https://www.blackhillsinfosec.com/team/ethan-robish/
About Ben Bowman - https://www.blackhillsinfosec.com/team/ben-bowman/

  • (00:00) - Intro: The Reality of Unsecured AI Agents
  • (00:24) - The Agentic Security Maturity Model Explained
  • (07:20) - Level 0: Total Chaos (Unrestricted Agents)
  • (11:24) - Level 1: Containment and Basic Guardrails
  • (13:24) - Level 2: Controlled Execution
  • (20:32) - Level 3: Monitoring, Logging, and Visibility
  • (27:00) - Level 4: Fully Enforced Agent Security
  • (28:00) - Final Takeaways: Maturity Over Hype

Click here to watch this episode on YouTube.


Brought to you by:
Black Hills Information Security 
https://www.blackhillsinfosec.com

☯️ Introducing BHIS Fusion Penetration Testing
https://www.blackhillsinfosec.com/fusion-penetration-testing/

Antisyphon Training
https://www.antisyphontraining.com/

Active Countermeasures
https://www.activecountermeasures.com

Wild West Hackin Fest
https://wildwesthackinfest.com

🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
https://poweredbybhis.com


Creators and Guests

Host
Brian Fehrman
Brian Fehrman is a long-time BHIS Security Researcher and Consultant with extensive academic credentials and industry certifications who specializes in AI, hardware hacking, and red teaming, and outside of work is an avid Brazilian Jiu-Jitsu practitioner, big-game hunter, and home-improvement enthusiast.
Host
Bronwen Aker
Bronwen Aker is a BHIS Technical Editor who joined full-time in 2022 after years of contract work, bringing decades of web development and technical training experience to her roles in editing pentest reports, enhancing QA/QC processes, and improving public websites, and who enjoys sci-fi/fantasy, Animal Crossing, and dogs outside of work.
Host
Derek Banks
Derek is a BHIS Security Consultant, Penetration Tester, and Red Teamer with advanced degrees, industry certifications, and broad experience across forensics, incident response, monitoring, and offensive security, who enjoys learning from colleagues, helping clients improve their security, and spending his free time with family, fitness, and playing bass guitar.
Guest
Ethan Robish
Ethan Robish has worked with Black Hills Information Security (BHIS) since 2008 — first as an intern and then as a full-time Security Consultant starting in 2012. In his current role as a Threat Hunter, Ethan is involved with customer engagement, research, working with Active Countermeasures’ AC-Hunter, as well as improving BHIS HTOC and SOC offerings. Previously, he implemented defensive security solutions for the Exchange Online security team as a Microsoft intern. While in college, he competed in the International Collegiate Programming Competition (ICPC) World Finals. In his time off, he enjoys cooking, playing the piano, and reading fantasy novels.

What is AI Security Ops?

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

Brian Fehrman:

Welcome everyone to this week's episode of AI Security Ops. So picture this, a developer downloads an AI coding agent, points it out to code base, and says, go fix these bugs. The agent has full access to their machine, their file system, their network, their credentials, and it's making autonomous decisions about what commands to run. No guardrails, no logging, no one watching. That's not a hypothetical.

Brian Fehrman:

That's how some organizations are running agents right now. Today, we're gonna walk through a five level maturity model for agentic security starting from level zero of total chaos up to level four four fully enforced. But before we get started, let's talk a little bit about Black Hills information security. If you or your organization are in need of any security service, whether that's external pen test, internal pen test, assume compromise, c two. What else do we do?

Brian Fehrman:

Web apps, physical pen test, wireless. We also have, SOC services, IR services, anything security related, check us out at blackhillsinfosec.com. Additionally, we offer training services through anti siphon training where all of our, or some of our practitioners, consultants package up their knowledge in a digestible and affordable way to share with you to help you level up your career, gain some new skills, or maybe pick up a new hobby. Check them out at antisyphanttraining.com. So let's hop in.

Brian Fehrman:

Someone wanna give a a little bit of background maybe, a little bit of level setting on why this topic matters and maybe some terminology here.

Derek Banks:

Well, I I certainly, you know, from, you know, teaching AI and InfoSec stuff, have had many folks asking me. I've had people texting me from other firms going, how do we secure this agentic Claude code thing? And my my response was, that's a great idea. We should ask Claude Code. And so that's kinda where I started with it.

Derek Banks:

And, you know, I think that I mean, I'm not claiming I came up with everything here, but this is essentially what Claude and I kind of situated at a while back ago. And then I started adding a little bit to it, but I think that most of us at at this point are still getting used to, like, what even is this I mean, it was just last fall when I was at Wild West Hacking Fest, and one of our senior testers, you know, had been trying to get used to AI. I won't say they were an AI curmudgeon, so to speak, but they weren't fully, like, on board with it. They had no idea that AI was more than a chatbot, that it could do other things. And that, you know, there's a whole, like, ecosystem, not just even the agentic stuff.

Derek Banks:

And so

Bronwen Aker:

Who was that?

Derek Banks:

Well, I don't wanna say on the on the on the webcast.

Bronwen Aker:

Tell tell me tell me afterwards.

Derek Banks:

I'll tell you later. You have

Bronwen Aker:

to tell me afterwards because I'm dying of curiosity.

Derek Banks:

Well, I mean, but I mean, you think about, you know, October 2025, yeah, a lot of people were still like their their AI experience was like, oh, I'm running Copilot at work and it sucks. So AI sucks. Right? Yeah. Unlike, you know, me where at that point in time, was kinda getting started with quad code.

Derek Banks:

And then, you know, you know, in November and December of last year in 2025, it really started to take off. And now we're like like Ethan said in February 2026, it's like, can't believe we're sitting here at dinner talking about the planned phase in Claude Code. Right? And so it it changed really quick. And so I think, you know, organizations are struggling to say, well, okay.

Derek Banks:

Now what do we do? I mean, we have customers right now in the CPT group that are, you know, asking us, what do we do? Everybody's asking for it. Should we just give it to them? And so that's that's kinda like, like, level setting, like, where most people are at right now.

Derek Banks:

Or if I install it, like, what are the risks?

Bronwen Aker:

Well, there's also a lot of confusion about whether it's what is what is a chatbot versus what is an agent. And for for people who aren't technically savvy, that can be a a difficult concept to bridge because they're used to interacting with chatbots. Chatbots have been around for years and years and years. The the difference, of course, is now so many chatbots are powered on the back end by large language models. And, you know, that changes some things in terms of the user experience and a lot on in terms of of the technical thing.

Bronwen Aker:

But the agentic thing is what a lot of people are tripping over, and they they are having trouble understanding that, no, an agent is something that will actually do stuff on your behalf. And in order to do that, you have to give it additional permissions. And the the thing that I that I have trouble personally when I'm talking to people about this is managing the FUD, the fear, uncertainty, and doubt. Because on the one hand, Agentic is wonderful, and on the other hand, agentic is terrifying, and for the same reasons. It's because in order for an agent to be able to do stuff on my behalf, I have to give it powers.

Bronwen Aker:

I have to give it permissions. I have to enable it and allow it to go and do stuff, and then I have to trust that it's not gonna mess this stuff up when it does this stuff. Did I miss anything important?

Brian Fehrman:

Nothing to do with the, that nondeterminism portion is is certainly important to, to point out there that that's, that certainly some of the risk is that, you'd you've, like, you just you you don't know for certain that it's gonna do what you want it to do. Right?

Bronwen Aker:

And Well, I think those who don't know what nondeterminism means, it means that you can ask the same thing three times and get three different answers.

Derek Banks:

I'm gonna ask Ethan, you know, you you work in our socket, BHIS, and have you know, we have a fair amount customers over there. What are they asking and saying about Agentic AI and and securing it?

Ethan Robish:

So I don't see so much about customers asking, But the I mean, there's definitely concerns, like, one one customer as part of an audit. So they were being asked, like, hey. How how are you using a AI? Like, have you integrated anything? Like, the auditor was asking them.

Ethan Robish:

And so then they transferred that question to us to, like, fill in the blanks for, you know, how how they're using it in a service that they're paying for, how we're using it in a service they're paying for. So there's there's some concern there. I definitely see a lot of alerts. This is kinda tangential, but it's funny. Like, we write rules to look for suspicious, like, command execution, things like, not not everyone is going and running like, you know, crazy PowerShell scripts.

Ethan Robish:

That's usually a sign of like, oh, either automation, which we can tune out, or, you know, a bad actor, like, executing a payload. But now we see Claude doing it

Derek Banks:

all the time. I'd say, some of the stuff Claude does is gnarly. I'm like, wow. I didn't even know you could do that. Holy crap.

Ethan Robish:

Yeah. So fun fact, Claude looks an awful like lot like a bad actor. And what happens if it becomes one? Like, I guess you call it a malicious insider.

Derek Banks:

Yeah. So the first step, I think, you know so level zero, we'll define as, like, basically, you install a Claude in YOLO. Right? Which look, I'm not gonna lie. I have Claude installed on my my host system at the moment.

Derek Banks:

But I also defer I do I definitely do the first protection recommendation, and that is to turn on the slash sandbox. And then what that does is it it it basically restricts Claude to, you know, allowed directories. And I think that by default, one is dot Claude. And it'll ask for permission to write or connect outside of that sandbox. Now this is not perfect by any stretch of the imagination.

Derek Banks:

And that actually happened to me again this morning. I asked Claude to do something, and without asking me, it's like, oh, the sandbox is on. Let me turn that off and move on. And so, you

Ethan Robish:

know I think there's a mono fallback mode. There's like three set and this is quite specific. There's Sandbox or talking mode.

Derek Banks:

There's different settings to the sandbox. But look, at the end of the day, if Quad can, you know, change the settings, Quad can change the settings. And we keep picking on I feel like it's like Band Aid or Xerox. Right? Like, all agents aren't Quad Code.

Derek Banks:

There's a lot of different ones out there becoming more and more popular. Codex has certainly, you know, become more popular. I've been messing around with one called Hermes, which is an open source agent. There's also OpenCode, which is another coding agent. There's some distinguishing between, like, personal assistants versus call call call coding agents.

Derek Banks:

Open call is another example of an agent. And you can almost now when people say agents, they also mean, like, harness. Right? Harness being the set of code that surrounds an LLM that allows you to interact with it and gives it access to tools and capabilities. And, so that that's, I think, you know, where we're at now.

Derek Banks:

How do you secure that agentic implementation, which, you know, quad code or codex is probably really mostly what we're talking about the enterprise setting at the moment. And so on the sandboxing thing and Ethan, you you I think we're we're recommending the same thing when we're talking about it is that if you're writing a project, if you're doing something with agentic coding, use Docker. Like, turn on the sandbox and then also use Docker because then your code will execute in a container that's not your host machine. I actually use I I usually use my agentic stuff in a container on a VPS. So there's that.

Ethan Robish:

Double. Yeah. So I I think this is a good segue to, like, talk about, like, kind of a framework of how how these different levels apply. So I'm gonna reference Simon Willison. He he's maybe most well known.

Ethan Robish:

He's got a a blog, a newsletter. It's great stuff. I recommend it. I subscribe. But he's reckon well known for coining the term prompt injection, first of all, claim to fame there.

Ethan Robish:

But he defined what he calls the lethal trifecta. And the trifecta is access to private data. So he's talking about an agent. The ability to communicate externally, and exposure to untrusted content. So yeah.

Ethan Robish:

The the lethal trifecta. Like, those those three things need to exist before you have a recipe for a bad time.

Derek Banks:

But by default, if I install Claude code, and I ask it to go off to a website and render that way, or analyze that website, I've just basically given it that lethal trifecta. Yep. Yep.

Ethan Robish:

So the different levels and the different strategies, like, we we've got it kinda defined in levels, but they're not necessarily mutually exclusive or like one above the other. Like, can layer them together and do so Derek talked about sandboxing.

Derek Banks:

That's level one sandboxing. So

Ethan Robish:

level maybe so sandboxing can serve multiple roles. Right? Like, it can restrict file system access. So what access to private data does it have? And that's personally my biggest concern and like what I use it for.

Ethan Robish:

So I use Docker and, you know, I have volume mounts and stuff. So most sandboxes could have some ability to also like restrict the network access. So Derek, what you're talking about, like, you know, you tell Claude to go out to a website, you you might, in your sandbox, be able to define hard limits like firewall or ACL type rules that say, hey, I you're allowed to get to this, but not not over here. And then there's the whole debate on on the allow list versus deny list by default. And that that kinda covers the second leg of the trifecta, like, ability to communicate externally.

Ethan Robish:

Like, you're at least defining that. You're intentionally defining it rather than giving it free rein. And then the exposure to uncut trusted content, I mean, honest I mean, it unless you are scrutinizing everything that goes into your context into your prompt, into what it's accessing, it's it's gonna be more and more common where you're gonna get unstructured. Or sorry, untrusted content, and possibly malicious prompt injection, like Yeah. I'm not

Derek Banks:

really sure that it's possible to entirely do that. Yeah. No.

Ethan Robish:

Not not unless unless you have a project that is completely self contained. Yeah. Like, find all the reference material, you you compile it, and you set up for a specific task. But then that's more like, you know, that that's a one time thing.

Derek Banks:

That's what we're talking about. Everything that goes into the context window, it's all your skills, everything that you've put in to call, it's like Yeah. A lot of work. It's all going into your prompt to the LLM now. And so I I don't know many organizations that I have moved past sandboxing into control, which would be our level two here, where we're gonna add things like tool allow list, permission hooks, configuration management, and dependency policies.

Derek Banks:

Now some of this is built in to things like Daniel Meisler's PIE and the personal AI infrastructure. There are security hooks in there. You know, the agentic platform that we built for externals, at Black Hills Edge case is, got, some banned commands, like, all every agent that runs a tool is scrutinized, and there's some commands that, you know, won't run. Like, Netcat's not allowed. Right?

Derek Banks:

It doesn't need Netcat for anything. And so, you know, removing files not allowed. I've actually had PI security hooks tell me before, sorry, I can't delete that. I it's a security hook. Right?

Derek Banks:

And so hooks are things that are are built into agents' claw code

Ethan Robish:

where it'd

Derek Banks:

be like specific times, they'll run and hook into that code. So when a tool runs, when it exits, there's like four or five different times during the agentic loop that it will try and call a hook. And so that's one way. Let's see. Another cont oh, go ahead, Ethan.

Ethan Robish:

The the I guess, just for comparison to the sandbox. The sandbox is kinda like putting a a bubble wrap around actually, there's a sandbox called bubble wrap, but like around your your thing, like around your execution, your your agent harness, And what Derek what you're talking about, I think, is more configuration that the agent harness makes available.

Derek Banks:

It's it's

Ethan Robish:

like tight more tightly integrated to like how it's calling the LLM and Yeah.

Derek Banks:

Really interpreting the responses. Too. And and something we didn't mention, there are a couple of different sandboxes out there. Microsoft just released one. Brian was talking about is a Microsoft product sandboxing.

Derek Banks:

Nvidia has one called Nim, I think. That's a sandboxing technology. I stuck with Docker because I've been using Docker for a while. And I just I don't know. I'm used to using it.

Derek Banks:

Right? I mean, if you want, and a lot of our pen testers do this, you could always just run it in a in a VM too. You know, that that that would count to sandboxing.

Bronwen Aker:

The only

Derek Banks:

thing I don't like

Bronwen Aker:

only thing I don't like about using a VM is that it it is you can't get access to GPUs that way.

Derek Banks:

Yeah. It's more difficult. It depends on what you're doing. Like, if you're, you know, with all of this, your mileage may vary in your abilities to do any of this stuff, or it doesn't even fit your risk profile like Ethan was talking about. I don't want it reading my my my secret keys, like, for SSH and such like that.

Derek Banks:

But there could be a reason. In fact, I have a reason. I actually let my agents have specific sets of keys because they need them to do things. Right?

Ethan Robish:

Oh, yeah. I do the same thing, like principle of least privilege, like Yeah. Exactly. Generated specific keys with specific permissions to give So the it's not like carte blanche, but it's, hey, here's the tool you need to do the job I need you to do, but no more.

Derek Banks:

Yes. For this on this specific VPS. Alright. So do it here. And and so but, you know, allow list, deny list, those kinds of things.

Derek Banks:

And I think that, you know, most organizations will start going down this road pretty soon in terms of using your existing configuration management to control agents.

Ethan Robish:

And the the control, I I mentioned, like, I don't see these as strictly, like, exclude mutually exclusive or specific one layers on top of the other. Like, you can combine them. So we actually when we started integrating or adopting age agentic workflows in the SOC, we started with level two. Like, we didn't have a sandbox. We skipped level one.

Ethan Robish:

We we started with configuration of Claude and putting in hooks and allow list, deny deny list, all all that kind of stuff for both network and for you you mentioned, like, what Py does, like, specific commands, like, hey, you can't run this command without approval. And that was that was baked into the the agent's harness, which then enforced it. And so Brian, you what do you use?

Brian Fehrman:

Depend depends on the situation, but I definitely have on like, it has to ask me permission before it does anything. That's that's one of my main things. And, in certain situations, I'll use, like, firewall rules. Like, if I don't want it talking like, if I'm gonna use a local thing and I don't I wanna make sure it's not gonna talk out to the network, then I'll put in, like, UFW rules to just allow it to communicate locally on that from that specific system to my local systems in the environment. Mhmm.

Brian Fehrman:

I think Derek mentioned this earlier, but if you if you're not

Ethan Robish:

a wizard in UFW or IP tables, you you can just ask the agent. Right?

Brian Fehrman:

Yeah. Exactly.

Ethan Robish:

Not necessarily to do it, but to like, hey, how would I do this? And then you you can look at and make sure that you don't see any glaring holes that it's putting a backdoor for itself in or something.

Derek Banks:

You know, it's funny. As a side note, I I've pretty much stopped saying, oh, I got Claude to do this. And I know I I I said that kinda earlier. I was making a point. But I think the terms are gonna go away, like vibe coding and I, you know, I did this with AI.

Derek Banks:

It's just gonna be assumed that everybody's using AI. And we had someone in our sales group say something this week that just kind of has has stuck with me. And that is shout out to Tom. Tom said, you know, this time last year, you know, all companies were coming to us and saying, you know, do you use AI? We don't want you to use AI.

Derek Banks:

Like, don't use AI on our stuff. Right? And now companies are coming to us expecting that we are using AI, wanting us to be disappointed if we're

Ethan Robish:

not Well

Derek Banks:

disappointed if we're not results.

Bronwen Aker:

Based on conversations I've had with CJ, everyone's coming in expecting that we're using AI. Some want clarification on what that looks like. And now, of course, as you say, some want us to use AI thinking that somehow the results is gonna be better than from using humans.

Derek Banks:

Well, I mean, I think that together, both are stronger. Right? And so now, I know we're segueing off the content of this.

Ethan Robish:

Yeah. I mean, it's it's the argument of, do you allow pen testers to use tools like scripts, scanners, vulnerability scanners? Right? Like, some say that was that's cheating. That's you're not fighting the vulnerabilities yourself manually with a, you know, magnifying glass modifying the I mean, you're not

Derek Banks:

writing your own custom shell code to find to exploit exploit the the zero

Ethan Robish:

zero a day. Right? It's the next it's the next level of automation. Like, helps you get better at what you're But already good it can amplify bad stuff too. So if you're not good at something, it'll just make you worse at it.

Derek Banks:

Definitely. Level three for this is monitoring, which, you know, in in network security and, you know, the defensive blue team end of of information security, you know, definitely logging is something that should be near and dear to everyone's heart. We should all, like, have public outcry for their lack of logging that comes along with AI platforms and tools. I'm still not impressed. Yes.

Derek Banks:

I do know that Cloud Code is a lot better at logging at least locally. Right? But, you know, I I would bet a beer that I there aren't very many companies at all that are centrally logging like all, like, called input and output in their environment. Actually sounds kinda hard to do. Right?

Derek Banks:

But should you do something like that? I mean, maybe. It depends on your environment, but at least logging locally if something happened would be where I where I'm at. So, you know, every agent the action takes is logged and and and shipped to a security stack would be the ideal scenario. Again, I'll talk about the thing that Brian and I've been working on, you know, for external penetration tests.

Derek Banks:

Like, every tool command is logged. We we start up a network capture to, you know, log all the traffic. So definitely, you know, I I think logging is important because if something happens, you gotta be able to go back and and and find it out. That that that's a incident response one zero one. Right?

Brian Fehrman:

Yeah. Yeah. I mean, at least at least logging, like, the the commands, even if if people wanna just take that as a first step before they get into, like, full, you know, query response logging, which which would be great. But, obviously, it's a lot of data, and there's a lot of logistics involved in being able to do that. But at the least, being able to, to look at the commands that were run by a particular agent for forensics reasons, and for anomaly detection, for any of the other reasons that you would log commands normally on a on a user workstation.

Brian Fehrman:

Right?

Derek Banks:

Well, I mean, if you ask me, like, you know, what should you do on an endpoint for monitoring and detection? One of the things I would say is you should turn on command line login. Right? And that's what we would say in our SOC, like deploy Sysmon, deploy an EDR that does it, whatever. If someone has to come back and do forensics, they have to be able to look at the commands that were run.

Derek Banks:

Otherwise, they're gonna say, yeah, don't know.

Ethan Robish:

We actually we do take a step towards this direction in in our SOC. So we use Lima Charlie. We build our SOC on Lima Charlie for logging. And Lima Charlie or probably one of the pioneers of this, but came out with a platform to to ship agent logs. So they've got an installer that hooks into a bunch of different harnesses, Cloud Code being our choice.

Ethan Robish:

But yeah. Every every tool call, I think even like the know, the input output for like what you're sending the agent might be logged.

Derek Banks:

I was trying to log a softball for you for, hey, we do this in our SOC. Right?

Ethan Robish:

Yeah. Yeah. The name, I'm not sure if it's gonna stick or not, but they it's currently branded as Vibreils. Like, kinda like guard Vibreails? Kinda like guardrails.

Ethan Robish:

But yeah. It's it's it's a logging telemetry platform, cent centralized. And then they actually you can write rules to take action and to, like, stop stuff from happening live as well. So I'll I'll tell an anecdote here. I was prompting so this was back before I used Docker.

Ethan Robish:

And I was doing what you did, Brian, and have it, like, ask me before it runs any any command. And so I was working in a git repo, and I was trying to get it to use the GitHub CLI to create a p p r, which it's done plenty of times without problem before. And that so that that was fine, but I wanted it to take this markdown file and attach it as a file to the PR, instead of like, you know, copying the content and putting it in a comment, because it was just it it was an artifact. It was long it was a long document. Didn't need to be like a comment.

Ethan Robish:

So I just wanted to attach as a file. And it couldn't figure out how to do that. So it decided on its own, like, I'm gonna I'm gonna create I'm gonna copy this content or this file to a a public gist.

Derek Banks:

Of course.

Ethan Robish:

Whatever. And and that and that that came up like, hey, would you wanna allow this? I was like, no.

Brian Fehrman:

No? No.

Bronwen Aker:

What part of no do you not understand?

Ethan Robish:

Yeah. That was its that was its workaround for not being able to figure out the the the right API or the

Derek Banks:

The command line. Command line. It's a clever workaround.

Ethan Robish:

Yeah. So after that, because I had five rails, like, installed the logging, like, went and wrote a rule to to look for that kind of thing happening and just to have it stop. Yeah.

Derek Banks:

My my first foray is into agentic coding. I found some things around on my hard drive, and I was like, why is this Python script here? Oh, because Claude decided he just wanted to write.

Brian Fehrman:

It's like, yeah, maybe we need

Derek Banks:

sandbox enabled here.

Bronwen Aker:

Well, I I told you about using Claude to deduplicate my music collection.

Ethan Robish:

Yeah. I haven't heard that. Go ahead.

Bronwen Aker:

Yeah. I I in order to teach myself Claude co work, I I had it helped me deduplicate my music collection. I mean, in the I mean, I copied all the files over into a sandbox area, and that was where we did everything. But I managed to save gigabytes of data and drop the total collection count from, like, 17,000 files to closer to 10 or 12,000.

Derek Banks:

Nice. That sounds like one of my Yeah.

Ethan Robish:

Sessions from Here

Derek Banks:

I was no. I'm kidding.

Ethan Robish:

I was I was expecting you to be like, yeah. It dropped all duplicates, but it deleted all the duplicates. Like, there was none none left.

Bronwen Aker:

Well, it was it was just co work. And it did ask permission, and then I needed to give it extra permissions in order to be able to to do any deletions. So

Derek Banks:

So the Yeah.

Bronwen Aker:

That was that was a fun exercise.

Derek Banks:

The last level, level four, would be enforced. And so that's really different than controlled where, you know, you have configuration changes that people are are are are, you know, that you're in control of. This is where you're gonna have like configs to push through your MDM or through like SCCM or something like that. You know, there's organizational policy, continuous verification, incident response playbooks, like, basically, the this is the mature end of the spectrum. Where you are with other things in your environment, you know, like Windows work stations, for example.

Derek Banks:

And so I think that this is the you know, Agentic AI is probably groundbreaking enough where a year from now, like, this is just where people will probably be. At least some organizations. I mean, think there are some organizations that are still out there with their Windows endpoints. Right? But I I think that this is the the the route that it's eventually going.

Bronwen Aker:

No lies detected.

Brian Fehrman:

No. I agree.

Derek Banks:

Alright. Well, so I guess, you know, clean way to summarize this. Level zero to one contain the blast radius where AI can act. That's your sandboxing. Levels one to two where you constrain capabilities like hooks, allow list, deny list, those kinds of things for commands.

Derek Banks:

Levels and from two to three is gaining visibility and and from you know, to see what the agent tool calls are, what the agent can do. And then level four is, you know, a mature security program with an enforcement of configuration on on your endpoints. Anybody have any closing thoughts?

Bronwen Aker:

RBAC is your friend.

Derek Banks:

Don't forget that whatever you have access to, if you install a agent on your computer, that that digital simulated reasoning entity that's up in the sky also now has access to the same data that you have access to.

Ethan Robish:

AKA Skynet.

Derek Banks:

AKA Skarnet. Exactly. We're in the early days. So be nice to your AI.

Brian Fehrman:

Yeah. That would please and thank you.

Derek Banks:

Thanks everyone for joining. I was waiting for somebody to say, keep on prompting.