Speaker 2: I really feel that input validation is a losing game with
AI because natural language has so many, like a natural language
system has so many bypasses.
And like, even if you block all special characters, like some of
the techniques, you can literally just describe them with words.
And so like there's always gonna be a creative bypass in this type of system.
Speaker: Simply Defensive brings you the industry's top practitioners,
innovators, and leaders to inform, educate, and join us defensive.
Josh Mason: Hello, and welcome to the latest episode of Simply Defensive.
I'm Josh Mason, the only person here who hasn't keynoted a Wild West
Hacking Fest, and with me as always.
Wait, what
Wade Wells: What's up?
Was last year.
All right.
And it was the, closing keynote.
I don't know if that counts or so
Jason Haddix: it does.
Wade Wells: does it.
Honestly, it was the better of the two.
Jerry's pretty upset about it.
Josh Mason: It was pretty damn good.
Uh, and with us today, uh, our good friend Jason Haddix.
Jason, thanks for joining us, man.
Jason Haddix: Hey, thanks for having me.
Josh Mason: And I feel like you need an introduction if you all don't know
who Jason is and you're listening to us, like choices, life choices so
Jason you you're running arc anum.
For those who don't know you, you've been the siz o at
UB soft.
You've been a bug crowd.
You're doing stuff with flare.
And you, in my opinion, are one of the thought leaders in
the AI cybersecurity space.
Just all Around.
you seem to have, a good grasp of how to attack it, how to use it.
. Wade, you have taken Jason's class on using ai.
Wade Wells: yeah.
I took the Red Blue Purple AI course, I wanna say maybe two years.
There's a second cohort that was going through.
I think.
Jason Haddix: been a year and a half ago.
Wade Wells: Yeah.
Yeah.
I saw you on Jerry's.
It's on what is the fireside chat?
And I was like, okay, I need, 'cause I had been playing around with AI for a while,
but I hadn't really dived into it yet.
And honestly the course just skyrocketed my experience like
Jason Haddix: to hear.
Wade Wells: ridiculous.
And it I came back and had instant like use, I'm still using the stuff
today at my current org and I am signed up for the course again.
And so gonna be another one in December.
We don't know when this is gonna air, but that's, to say.
I do wanna shout out one thing though, in that class way back then, you were
talking shit about CTI and Little bit.
A little, and now you're a field CISO for flare, so I just
thought that was hilarious.
But.
Jason Haddix: So I think the con, I think in that class, the conversation
was that I've had a lot of different hats and I've had to manage a lot of
different groups inside of security it was always hard as a leader.
To prove value of CTI.
Not that I didn't think that CTI was valuable.
It's just hard for them to prove what we're doing every day is cost effective
as opposed to other groups in security, which give you an artifact pretty much
after they do anything right, like pen
Or And I think CTI and threat hunting suffer from this both a little bit.
And sometimes your CTI people are your threat hunters.
It depends how big your organization is Right?
some teams wear many hats.
But I think that was one of the things it's it was just like I remember.
I remember looking at some investigations and then some threat hunts and then
looking at artifacts and it just wasn't where I wanted it to be.
And that could have just been my personal experience.
It's every, place is different.
But like I do think that there are some very pointed, kind of components
of CTI that are really valuable.
So the cyber I think that.
The credential kind of leakage problem.
Wade Wells: yeah.
Jason Haddix: Is one of the biggest ones.
If you look at any fucking breach, oh, sorry.
Wade Wells: No, we can cuss.
Go for it.
Josh Mason: You're fine.
Wade Wells: You're good.
Yeah,
Jason Haddix: Yeah.
If you look at, if
Josh Mason: We're grownups.
Jason Haddix: Across the industry it's like something that's 85% of
'em are started by some sort of either leak credential or someone's
been part of a different breach, password reuse or something like that.
And so like that whole part of.
is really important is to first of all, identify which accounts have been
leaked, how they have been leaked, what the mitigation is going to be for that.
Whether you're going to roll a credential, whether you're going to
reset a password, whether you have to reformat a whole goddamn machine.
'cause there's malware on that machine,
right?
Wade Wells: Yeah.
Jason Haddix: And I actually see that last one I don't see
a lot of people talk about.
Like when as part of Flare, I've learned a lot about the CTI underground,
and when you get Steeler logs and you look at those and you're like,
okay, an employee's laptop has been infected with actual Malware.
that, that exists usually at like the system level,
right?
Like on the kernel level, depending on they had an
exploit or something like that.
And so a lot of customers like, oh, we'll just roll the account.
It's.
Wade Wells: No,
Jason Haddix: you have to like like actually nuke that system and start from
Wade Wells: Okay.
Jason Haddix: And that, that's a harder discussion too, if it's a personal system,
Josh Mason: Oh yeah.
Jason Haddix: someone's
Wade Wells: Yeah.
Jason Haddix: that they logged into corporate resources with,
how do you go tell a personal.
Per oh, your, the laptop that you and your kid and your wife uses at home got
malware on it because someone downloaded a crack for Fortnite or free B bucks or
whatever, and I need you to reformat your reformat that box because it's it's got
malware on it that disclosed corporate credentials or cookies or whatever.
Yeah.
Wade Wells: To your point to to give you a little more credit.
So I did raise my hand up and I was like, oh, I'm the CTI instructor at Antis
Siphon, and you did let me come on and defend CTI and then you, were, all for it.
So
Not more of just poking fun, but with, definitely with the.
Imaging stuff.
Now, especially with more remote work, like I, when I, one of the big
corporate overlords I used to work at, if there was any signs of malware, it
wasn't even a question like, oh, we're gonna get, no, you just go get a new
PC where we're wiping you complete.
Nowadays, I don't think I've seen that since remote work at
all, at any of the orgs I'm at.
And it'd be much harder, like I've been nuked twice by antivirus
'cause of doing funny stuff and they have to send out a new Laptop.
Jason Haddix: I think Josh, was it you that I was talking to at Wild West and we,
were talking about the struggle to even get machines at the beginning of COVID.
Was that you
that I
Josh Mason: No, but I, recognize that I saw that myself.
Jason Haddix: There was a little while where we like we, I was a
cso, UB Soft, and we, we need.
needed machines for new employees.
And during COVID, during the first year and a half, Dell
wouldn't even sell us anymore.
And so what it meant was everybody went home to work and we had new employees
still coming on board during COVID.
And now for a little while it meant that we asked them to
use their personal devices,
Wade Wells: Wow.
Jason Haddix: were pre-owned.
Wade Wells: Yeah.
Josh Mason: Yeah.
Jason Haddix: Yeah, so it's, not an easy question with, that blip of
three years, three and a half years too, that that happened to where
everybody like moved to remote work.
It's, an infinitely hard balance to, do.
Yeah.
Josh Mason: Oh, so much I was a government contractor at the time and there was
so much of, you definitely can't log in from your personal and to do stuff.
That was a real difficult thing to do.
Yeah.
Jason Haddix: Yeah.
Wade Wells: I wonder, I never even looked at, so I wondered if the
VDI market like spiked then, right?
Josh Mason: Oh yeah.
Yes,
Jason Haddix: thing we looked at.
Wade Wells: Yeah.
Josh Mason: a hundred percent because that was our
Jason Haddix: we
Josh Mason: solution.
Jason Haddix: video.
Wade Wells: Oh, okay.
Jason Haddix: Yeah.
We, I mean that we had so many employees that we had 22,000 employees,
so the V, so any VDI solution that we would've pivoted to first of all
would've taken forever to implement.
And then second of all, would've been so expensive and crushed our
security budget and our IT budget.
And
Wade Wells: And then you gotta hope those,
Jason Haddix: it out.
Wade Wells: you gotta hope those VDIs are then set up properly, right?
Jason Haddix: There's management and like
Wade Wells: yeah.
Jason Haddix: and all that
Wade Wells: Use a different golden image and you're like, wait a second,
Jason Haddix: Yeah.
Wade Wells: you're not supposed to have access to that.
Yeah.
Jason Haddix: Exactly.
Yeah.
Josh Mason: Yeah,
Jason Haddix: let's pray that someone doesn't put a hard coded admin account
on there and it's got a hash and then some low privileged user just
pulls that sucker out there and Yeah.
Josh Mason: the uh, instructors were happy because finally we had computers
that we could mess with just for fun, that had Cali because we were
using vSphere, and they just spun, or they just turned around and used the
golden images that we had for classes.
Just took those pools and just aot of them to us for usage
to then log into the courses
To allot them to students.
So it was it was, I don't know.
I wasn't in charge of it.
I wasn't the vSphere admin, thank Goodness.
yeah.
But that was a mess.
Wade Wells: All right.
I have a good AI question for you.
And when we discussed here before.
So me as like a blue teamer, right?
I've seen like the, Soar come up, right?
And everyone's we're gonna automate everything.
Everyone's gonna leave.
No one's gonna have a job 'cause we're gonna be able to do it.
I felt like in my, career, people did use Soar, but everyone was still
scared to do like manual pushes and, blocks and stuff like that.
Like I'd never seen someone go full bore with it.
Jason Haddix: Okay.
Wade Wells: I feel like people are thinking they're gonna do
the same thing with AI now.
But if like people didn't even use SOAR for that, why
are they gonna trust AI more?
Does that make sense?
Alright.
Jason Haddix: I think that, in the first two cohorts of attack
of of Red Blue Purple ai which people don't know what that is.
It's a course that we run we spend one day getting you from
zero to hero to understanding the current kind of models and ways
that you can consume those models.
And then we spend a day talking about how to apply it to security roles.
And and we look at some of the more cutting edge.
Projects out there that are already using AI for blue teams,
red teams, and purple teams.
And so I feel like in that the the first few cohorts, I was very bullish on that
it would happen very fast that we would.
We would automate some of the detection work, some of the glue
that puts those systems into other change management systems and asset
systems and all this kinds of stuff.
And it would really happen very quickly.
Now since then, we have been like doing some consulting, going to organizations.
As part of this new random service that we built called the an AI
scaling Assessment where we go into a business and we're like, let's
look at everything you're doing in security from a security point of view.
And then I will, basically consult with them and say, here's where
you can use AI for these teams.
what I have started to learn over the course of doing these
assessments is that, organizations move gally slow for change like this.
And so I knew that from being a CISO in certain orgs, but it's even more
apparent when you work with companies in the finance sector or in the
healthcare sector or the automotive sector, which some of our, which
are some of our biggest customers.
And so Like it, it just, I have had to slow my role a
little bit and be like, cool.
Although the capability exists for you to do some of this cutting edge
stuff, like using MCP to use natural language to execute investigations
or to build custom dashboards or all this amazing stuff, it all exists.
With a couple of with a couple of vendors and some open source stuff,
it's in order to get that into any of these organizations, it's gonna
take a year and a half, pretty much.
Wade Wells: Yeah.
Jason Haddix: at minimum.
And so that was something I've been learning over my time with AI is
that even if the tech is there, I.
Sometimes the businesses are not ready to move, the security
teams are not ready to move.
And that, that's just the technology portion of it, like the implementation.
There is also people inside of these organizations who are anti ai.
And I've had a lot of this in my interviews with these security
teams and they're like, I.
They don't, first of all, they have some latent fear of being replaced.
Which is normal.
It's human.
And that's totally a thing.
And then also they've had bad interactions with their usage of AI where they,
no one had really trained them how to use any of these models really well.
So they tried to use it for something and it didn't work
well, like the first two times.
And then they were like, F ai, is so junk.
Like it's never gonna take the place of anybody.
And then you have other people who, Just like they inside of political,
inside of political organizations, they try to protect their fiefdom from
change because they have a well-oiled machine in operations or in the blue
team or in the sock or whatever.
And they finally have just gotten their product that they wanted in there.
They finished all that onboarding and they don't want to change any part of
that 'cause it basically gives them mental anguish to think about doing any change.
Yeah.
Wade Wells: I recently, okay, so I'm at a newer org.
I meant one password.
I've been there for almost coming up a little over six months.
Yeah.
Thank you.
Jason Haddix: Yeah.
Wade Wells: It's pretty fun.
We're right off the bat, so we're very project oriented project and like
making issues and stuff like that.
And for security, like I haven't seen.
close of like more of a developer life cycle than anywhere else.
Jason Haddix: Yeah.
Wade Wells: right off the bat, my boss tells me is Hey, your issues are lacking.
Like you need to be more verbose.
You need to be better, like what you're gonna do with them.
And I'm like, oh, okay.
didn't make new issues.
I just made a, I made a, good bot that to I give it the idea, here's
the template and then the, best part is he came back to me, he's
dude, you're rocking it on issues.
And I'm like, oh no, I made a bot for that.
Here, it is.
And he is just give this to the team.
Everyone's just gonna use this for issues now because everyone else.
So stuff like that
Jason Haddix: Yeah.
Wade Wells: just I'm not seeing enough people that call out the little
things that you can use it, that are
Josh Mason: Oh my gosh.
We're huge on that Here.
our big thing, man, over the summer was like, if you we have, you know,
a, an enterprise AI for like everyone to use, like use that one for work.
And, use it for as much as you can.
It's got all the features, so like that puts all the fences on it that
I don't know, someone signed off On.
and, uh, the CT like CTO and CEO signed off on it.
And so it's got all the features, we've checked it out, like it's
got the fences, so thus, use it.
And for the small things, for the big things.
I ran deep research like 20 times this week
On like crazy ideas and I'm, yeah.
Because I got, I'm taking over like the company podcast as well, because
Jason Haddix: cool.
Josh Mason: you can never have too many podcasts, in my opinion.
Jason Haddix: Everywhere.
Just Josh, you're crazy.
Josh Mason: I'm bored and
Jason Haddix: You talk to my wife more on signal than you talk to me.
Josh Mason: whoa, whoa, that's gonna, that's gonna be weird.
That is not true.
Hi, Julia.
You're great.
Jason Haddix: and Julia hi.
Wade Wells: Yeah.
Josh Mason: That does sound really weird.
You gotta admit.
Yeah.
The autistic in me is just like that.
I don't know how to respond to this.
Okay.
for everyone who's listening, Jason and Julia are really awesome and, uh.
Yeah, we're all good friends.
But Jason's really freaking busy and so Julia handles scheduling anyways.
Um, okay.
Whew, way to ask something.
Wade Wells: Oh man.
Man, ask something.
What do you think about I've been looking at all the good, like the new
detection stuff that's been coming out more for ai, like detections.ai
we had like their field ciso, Aaron on and stuff like that.
What do you think about those for right now?
Do you think that's just natural progression?
Jason Haddix: I think
Wade Wells: Yeah.
Jason Haddix: I think that the I wrote an article on this a while back and
it was like the, it was also from this experience of going into orgs
and talking to 'em about ai and I feel like there's there's an adoption.
Ladder or something like that, whatever you want to call it.
Stages of adoption and they will be blocked by different things.
But the stages are, really clear, at least in my mind.
And it's okay, so I mean you remember when you took Red, blue,
purple AI and the first thing we talk about is just building A GPT
Wade Wells: Yeah.
Jason Haddix: how important.
System prompts are right.
And we teach a custom prompt engineering methodology in that class.
And I've talked about it in talks before, but I really haven't released
it or anything other than in the class.
But but prompting is really important.
So like most orgs they have access to, like what Josh was saying was like, okay,
we buy access to a co to, or we have Microsoft and we've turned on copilot
everybody so everybody can make Copilots.
So the first kind of adoption is building what we call custom bots, right?
Which is just adding a system prompt to do a task or to build a
bot to do a single task in copilot.
And so that's like, level one adoption.
Like at the end tail level, one odd option you might add rag to it, right?
Where you add custom documents or data sources to that bot.
So it can basically have answers to questions that
are not in the training data.
And so that's at the end of your kind of like stage one journey.
And then you move on to stage two where you do things like agents where you break
out a whole system which has access to individual little bots that do individual
things and bring them back into a plan.
and I think right now the general industry at large is still at the tail
end of probably that first stage where you know, people are starting to learn,
oh damn, like AI is now good enough to write detection, engineering rules
in my favorite tool, or help me build signatures or even help me like make
my EDR better by like doing custom stuff that was never available to me.
I see this in the red team side all the time.
It's like I see red teams who didn't have the money to.
As part of their team, have a tool developer who did custom tooling
or build custom phishing frameworks or do all this stuff that they
just, they couldn't do it before.
And now with ai, they're able to do a lot of this stuff which is, really cool.
Which often requires a lot of knowledge of some C two or whatever.
And yeah, so I see a lot of people at that end stage right now in, in the industry.
Wade Wells: I haven't seen as much talk about or at least in the blue team
phase, like I don't feel enough people talk about like how essential using rag.
Is in order to like really build yourself up.
Jason Haddix: Yeah.
Wade Wells: I've, I have pretty much profiles of my entire company.
Our entire tool sack, our logging, how, what's the fields in those
logs provided that to a bot.
And now when you wanna build detection, it tells you exactly what logs, the query,
the, and it's amazing that I I don't know why more people aren't doing it, or people
at least explaining the customization at some of these like blue team talks, but.
Jason Haddix: I think that the domain has gone past prompt engineering to now
what we call context engineering, right?
Which includes rag, it's
The whole context engineering thing is, it's a really simple idea.
It's like how do you add information on top of the model?
To make the purpose of your bots or your your API call or whatever.
Way
better.
One of the classes that we're building right now is not
actually a security class.
It is a class just on that.
And it's
Josh Mason: Wow.
Jason Haddix: do context engineering.
And it's like rag is one you said, right?
And How to do really good prompt engineering.
And what we did is we went out
Josh Mason: Stop.
Jason Haddix: at all of these AI first companies and eventually
their system prompts get leaked on the internet and to GitHub.
And so we started reverse engineering them and being like, okay, this is
how they're calling tools reliably.
This is how they're structuring order of operations for.
The ai, this is how their agent architecture looks like.
This is how their rag looks like where they're pulling
it, how they're pulling it.
And then we also did that from the model vendors as too.
So like the model vendors have their system prompt, right?
There's two layers of system prompt that you know are in.
Anytime you use open ai, there's the.
the one set by OpenAI, and then there's also the one set by the
user, and then you know, the use and then the, or the developer,
and then the user chats with it.
All of the model vendors system prompts have also been leaked.
So we started looking in there too and being like, okay, what
are the best practices that they use to make these bots better?
And so it is I don't have any benchmark data or ever whatever, but after doing
all that research and and implementing some of those things like rag I think
in, even in the second cohort, we might have said examples of what you want
output to look like is really important,
like
Wade Wells: Yeah.
Jason Haddix: examples.
I think we, we said that in the course and and just like structured system prompting
and like tool calling URLs, all this stuff and like metadata fields that you can add.
It's like it's night and day difference.
It is I
Wade Wells: Yeah.
Jason Haddix: just in my head, like a 20 percentile effectiveness,
like increase from just.
Asking a straight question to the model
Wade Wells: Oh yeah.
Yeah.
Have you seen any other ones telling their bots to use methamphetamines?
Jason Haddix: I have not for a little while.
Yeah, so what he is referring to is one of the is one of the tricks in, I guess
it was prompt engineering at the time, but there was some white paper studies about.
Urgency prompting of which one was telling your AI that it was on methamphetamines.
And so that was one of the, what we called in the course, we called
it like silly machine tricks,
Josh Mason: Ah.
Jason Haddix: that we have in our methodology.
I still use that today and I don't know if my bots that I published publicly would
be as good as they are if they didn't have all those little tricks in them.
Honestly.
And
Wade Wells: Yeah.
Jason Haddix: couple new ones I think in the class that, that we have added
but we have also removed a couple too.
Yeah it's a evolving field.
Sometimes the models get good enough, you don't need things like that anymore.
And it's hard to know when too, to really, to remove the silly machine
tricks because the the you have to do a bunch of benchmarking in order to figure
out, like if it's having an effect.
And sometimes we don't have time to do that benchmarking, like right away,
Josh Mason: yeah, I have like my stupid trick with GPT five
is think hard about that.
So that it'll use the pro and then, tell it to go out and search the internet
to find the, find examples so that it won't just like hallucinate on whatever
it's trained and it'll go find proof.
Uh, when you were talking about your new course, I was like, oh
man, I really want to do that.
And then it hit me that, uh, I'm going to learn how to do that and
I'm just going to use it to use a rag to work on my DD campaign.
And.
Wade Wells: Oh, yeah.
Yeah.
You don't even need a DM anymore.
You just feed it all to the, bot, and then
you're
So and
Josh Mason: look.
Wade Wells: What's next?
Dude?
Josh Mason: No, Nah.
It, oh.
Wade Wells: good enough.
It's good enough.
You just drop, you go get a PDF of
Jason Haddix: the,
Wade Wells: The
Dungeons Master Handbook and drop it in.
Jason Haddix: Yeah.
Yeah.
Wade Wells: one, one aspect that I have I, know is giddy is starting to be a little
bit more a little bit more mainstream is like defending your prompts, right?
And making sure people can't reverse engineer them.
Jason Haddix: Yeah.
Wade Wells: to give an example we all know FedEx.
I have a well-known bot that does the Palantir's a DS framework and you to
give it a detection, it completely fills out the whole framework for you.
And then one day FedEx came to me, he is Hey, here's your exact prompt for this.
And I'm like, not upset about it but should I be protecting this?
And he's yeah, man.
This is your thoughts.
This is, I'm like, ah, all right.
What do
Josh Mason: That's a good point.
Yeah.
At what point does a prompt become ip?
Jason Haddix: yeah.
So this has been a discussion since the beginning of.
Of like basically the GPT store and and I think a lot of people originally
thought, oh, the GPT store on OpenAI.
It it's basically a user sets up a system prompt and then publishes a bot.
And so a lot of people have done it and originally they thought they
were gonna monetize those bots.
But think that prompt injection being so to use and there's so many bypasses
to any security control, you even put in the prompt that, you know in when that
simplistic model of like just a user interacting with one bot and there's no
classifier or guardrail in the middle or anything, system, prompt level,
detect protections are like the least effective out of everything basically.
And so there's always gonna be a way to.
To dump the system prompt.
And a lot of the times when you're thinking about defense of system prompts
or just defending an AI system in general I think I tweeted about it this morning.
It's like you have to shift away from your normal security mindset of like
input, validation, because that's where we all go immediately, right?
Wade Wells: Yeah, I was
Jason Haddix: app
Wade Wells: a AI waf, right?
Jason Haddix: And You're like, oh I'm gonna use I saw some guy no, no
fault of his, he's a great researcher, but I saw him post on LinkedIn
and he is he's this is just the same problem we had with web apps.
And it's we're just gonna make sure that special characters can't
go through a system or whatever.
And so I was trying to think about oh, he comes from.
The web app world and like input input malicious input, detection and then
output and coding are the the two big things that you learn when you're in
web apps and, even other protocols when you're hacking other types of stuff.
But I really feel that input validation.
Is a losing game with AI because natural language has so many, like a natural
language system has so many bypasses.
And like even if you block all special characters, like some of
the techniques, you can literally just describe them with words.
And so there's always gonna be a creative bypass in this type of system.
And so really I feel like a lot of people need to move.
Move towards like classifiers and guardrails in line with your AI system.
And those are basically sentiment analysis to see on the output.
It's okay, does this look like a normal response?
We usually give the user yes, no.
Okay.
Return it to them or Yes.
Return it to them.
No, it has like.
Random agent data, it has all this stuff, no drop, connection or whatever.
And I think that is a mind shift that a lot of the security people
are gonna have to make in, this era.
But of guardrails, classifiers, and prompt based protections, prompt
based protections are the weakest.
Usually you're able to bypass them.
I still think they play a role in defense in depth.
If you have all three of those and you are also doing several stages
of routing for a user question does it satisfy these requirements?
Does it contain these keywords?
Okay, then I'm gonna route it to the LLM instead of just directly to the LLM.
If you have all four of those things.
really hard to crack.
I've been up against a couple systems really recently that
had all four of those things.
So LM based routing that was contextual.
A classifier, guardrail and system prompt defenses.
And it was an extremely hard test.
Extremely hard yeah.
Josh Mason: Wow.
Wade Wells: I love the defense in depth reference.
That's it.
Like I, I ha The only thing I have thought about that too is exactly
what you said, like the web app, the waf just like monitoring that.
But all those other parts that's, pretty crazy.
Josh Mason: Yeah I've got one of our researchers who I get to
interview about a zero day that.
He gets to publish on Tuesday and I'm, excited.
It's fixed.
He was demoing it to me and all of a sudden like it stopped working.
I was like, oh, sick.
He's got videos of it working.
Wade Wells: Good.
Josh Mason: Yeah.
But it's pretty wild.
Yeah, and it's good that it's fixed.
Jason Haddix: I was talking to someone the other day.
I was telling him that one of our researchers on the team, he found
like on one of the, I'm not gonna say which one, but one of the
biggest model vendors out there.
He found a mass assignment vulnerability, which is I think the first time in, an API
chat completion or an AI chat completion.
API I think has had a mass assignment vulnerability in that kind of structure.
And so I like to tell people we're still in the infancy kind
of stage where we're finding bugs with these systems at every layer.
And it'll be like that for the next probably year and a half, two years.
And it's a great place to do research if you, like doing research and you
wanna dive into something, there's a whole stack of stuff you could look at.
You could look at the models, you could look at all of the.
Apps around the models you can look at prompt based detection,
prompt based hacking prompt injection methods, like there's so
much research going on right now.
It is so cool to go to the conferences and see so many talks yeah.
Josh Mason: Where's.
Wade Wells: we're at a time.
Are do,
Josh Mason: You asked.
First, you ask yours and then I'll wrap up with mine because it'll sound.
Wade Wells: we end the podcast on one question, whereas what's one piece of
advice you'd give a blue teamer right now?
Could be someone just starting out, someone with years of experience.
Doesn't matter.
Jason Haddix: I guess it's to embrace this new tech, right?
I think that I meet a lot of people who, you know on, whatever side
blue or whatever, but who just are not ready to accept that this is
a mainstream piece of technology that we're all gonna have to use.
I, fear for people who are very.
to adopt it just as a tool.
You don't need to make it your whole life, but understand that it is a
powerful tool that can do small functions of your job really well and make you
faster.
I don't see many complete automations of people out of their jobs right now.
That is the narrative, but I don't see many of 'em right now.
There's still human in the loop.
over the place, but it does make good people fantastic.
And so embrace the technology.
Take there's hundreds of getting started with AI classes out there that
are completely free YouTube channels.
Just learn how to use some of the front some of the frontier models.
And then as like we do in the class, right?
Take your job and think about what are the things I do day to day?
And then rank them about like, how annoying they are to you.
It's like what, in this.
like super annoying or requires me to like transpose data or like glue together
two systems and then just start chipping away at those micro problems using
AI and you'll become like a rockstar.
You just.
You just do, you build tools.
You have a bot that helps you do things and then you share it with your team.
And that, and then when like later on we do get to some automation stuff or
people are getting displaced maybe, which I don't think we're, it's
gonna be like crazy, but if it does happen, you're the person they call.
They're like, oh yeah, you made that bot, you're a, you're our
AI guy on the blue team, right?
So, embrace the technology, it's a tool.
Don't buy into the hype that it's gonna replace everybody and use it as, you Can.
Josh Mason: Love it.
My question is
Wade Wells: last, wait, one last piece.
Josh Mason: okay.
Okay.
Wade Wells: watch
Josh Mason: yeah.
Yeah.
Wade Wells: on YouTube.
Like
Jason Haddix: Yeah.
Yeah.
Fire is a great content creator around tech and ai.
Yeah.
Wade Wells: When, you, I like from that class.
I learned about him from that class and I've been watching it from then on, but.
Jason Haddix: He's great.
Yeah.
Josh Mason: I love that.
I love that.
Where can people, uh, keep up with you?
What's the best place to, uh, know where you're at, what you're doing?
Jason Haddix: So I am at J Haddix on Twitter.
So like my personal ramblings and sometimes our company stuff goes there.
But our website is ARKanum, A-R-C-A-N-U m-sec.com,
and we have a blog there.
Where we talk about research, we also have some resources there.
We have a GitHub where we have recently been pumping out tools and
resource sheets for free for everyone.
So if you find the AR, canem security ar canam information security GitHub,
there's a bunch of resources on that now, especially on GitHub pages.
So you can check us out there.
And then we have a newsletter too, which is it's a newsletter
called Executive Offense.
And usually if we have a big tool release or a resource cheat sheet
or something we're releasing, we'll release it on the newsletter.
So if you find the Executive Offense newsletter written by Jay Haddocks,
then you, won't ever you'll always see what we're putting out there.
Josh Mason: Awesome.
Those will all be in the show notes.
So if you can hear it or see this you can find them There.
Jason, thank you so much for joining us.
Uh, hope to see you soon.
Play some d and d, play some, uh, magic with you, uh, or at
least have a drink and hang out.
Thanks, Ben.
Bye y'all.
Wade Wells: See
Josh Mason: you, Wade.