AI Security Ops

In this episode of BHIS Presents: AI Security Ops, Bronwen Aker and Ethan sit down with Josh Mason for a wide-ranging conversation about cybersecurity careers, AI, small business security, and what it actually takes to help organizations make practical security decisions.

How do small businesses think about security when they do not have a full-time CISO?

And what happens when AI starts lowering the barrier for research, planning, governance, and security operations?

Not hypothetically. Not as a buzzword. But in the real world — where companies are trying to understand SOC 2, HIPAA, incident response, cyber insurance, AI governance, hallucinations, and the risks of letting tools make decisions they do not fully understand.

Josh brings a practical perspective from his background as a C-130 pilot, cyber leader, instructor, sales engineer, vCISO, consultant, and founder of Noob Village at DEF CON.

We dig into:
- What Noob Village is and why DEF CON needs an on-ramp for new people
- Josh’s path from Air Force pilot to cyber leadership
- Why communication and translation matter so much in cybersecurity
- What a vCISO actually does for small businesses
- How smaller companies think through SOC 2, HIPAA, GRC, pen testing, and incident response
- How AI can speed up research, planning, and draft creation
- Why AI-generated work still needs human review and source validation
- How companies are trying to govern employee use of AI tools
- Why cyber insurance, E&O coverage, and AI hallucinations are starting to overlap
- Where RAG and guardrails can help reduce risk
- How AI may reshape the work small businesses can do on their own
- Why trust, relationships, and human judgment still matter in consulting
- How hacker community, mentorship, and D&D all somehow fit together

This episode explores a practical shift in AI security: AI is not just changing the tools defenders use. It is changing how small businesses learn, make decisions, evaluate risk, and decide when they need expert help.

The takeaway: AI can make security work more accessible, but it does not replace experience, judgment, validation, or trust. The organizations that benefit most are the ones that use AI to accelerate good decisions — not outsource thinking entirely.

Chapters
  • (00:00) - Meet Josh Mason
  • (01:27) - Hacker Summer Camp and Noob Village
  • (06:45) - From Air Force Pilot to Cyber Leadership
  • (13:08) - What a vCISO Does for Small Businesses
  • (19:55) - Using AI for Research and Incident Response Planning
  • (24:46) - Small Business AI Security and Governance
  • (27:47) - Cyber Insurance, Hallucinations, and Guardrails
  • (31:43) - How AI Is Reshaping Small Business Security
  • (42:15) - Where to Find Josh

Click here to watch this episode on YouTube.


Brought to you by:
Black Hills Information Security 
https://www.blackhillsinfosec.com

☯️ Introducing BHIS Fusion Penetration Testing
https://www.blackhillsinfosec.com/fusion-penetration-testing/

Antisyphon Training
https://www.antisyphontraining.com/

Active Countermeasures
https://www.activecountermeasures.com

Wild West Hackin Fest
https://wildwesthackinfest.com

🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
https://poweredbybhis.com


Creators and Guests

Host
Bronwen Aker
Bronwen Aker is a BHIS Technical Editor who joined full-time in 2022 after years of contract work, bringing decades of web development and technical training experience to her roles in editing pentest reports, enhancing QA/QC processes, and improving public websites, and who enjoys sci-fi/fantasy, Animal Crossing, and dogs outside of work.
Guest
Ethan Robish
Ethan Robish has worked with Black Hills Information Security (BHIS) since 2008 — first as an intern and then as a full-time Security Consultant starting in 2012. In his current role as a Threat Hunter, Ethan is involved with customer engagement, research, working with Active Countermeasures’ AC-Hunter, as well as improving BHIS HTOC and SOC offerings. Previously, he implemented defensive security solutions for the Exchange Online security team as a Microsoft intern. While in college, he competed in the International Collegiate Programming Competition (ICPC) World Finals. In his time off, he enjoys cooking, playing the piano, and reading fantasy novels.
Guest
Josh Mason
Josh Mason provides strategic guidance and technical expertise to help organizations protect their data, applications, and people from evolving cyber threats. With extensive experience spanning cybersecurity, organizational development, and business strategy, Josh has advised Fortune 50 enterprises, government and military leaders, and emerging startups on strengthening their security posture. He has developed and taught courses covering secure software development, offensive and defensive cybersecurity, and risk management, and has demonstrated security solutions to executive and technical audiences around the world.

What is AI Security Ops?

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

Bronwen Aker:

Welcome to AI Security Ops podcast where we cut through the hype and explore real world intersection, the real world intersection of artificial intelligence and cybersecurity. Each week, we examine how AI is shaping, reshaping both sides of the security landscape, threats we're facing, and the defenses we're holding. I'm Bronwen Aker, and in this episode, I am delighted to introduce my very good friend and occasional cohort, mister Josh Mason. How you doing, Josh?

Josh Mason:

Hey. I'm doing great.

Bronwen Aker:

Awesome. So this show is brought to you by Black Hills Information Security and Antisyphon training. BHIS helps organizations close real world security gaps through penetration testing, adversary emulation, palpatine, managed detection, SOC as a service, man, we've got it all. Antisyphon on the other hand delivers hands on practitioner led training built around real attacks, real tools, so that when you take an anti siphon training course, you can apply what you learn immediately. Learn more at blackhillsinfosec.com or and or antisyphontraining.com.

Bronwen Aker:

And with that, the bills are paid. Josh, how the heck are you? What are you up to, my man?

Josh Mason:

I am crazy busy right now with DEFCON Black Hat prep. Hacker Summer Camp is a few weeks away, and I'm giving a talk. I'm introducing speakers of Black Hat, and I'm barely in charge of a whole community at DefCon for the second year, which I have to keep pinching myself that I'm in charge of this.

Bronwen Aker:

So what all are you doing at DEFCON?

Josh Mason:

So I started Noob Village, which is a, in DEFCON terms, a community. Everyone's familiar with The Villages. Communities are their term for smaller groups that are focused on something that's not like social engineering, red team, blue team, AI. It's for noobs. So not knowing that they were gonna come up with the that community vernacular, I started the nonprofit Noob Village, and they keep telling me don't use Village when I'm talking about the community.

Josh Mason:

C'est la vie. But it's a lot of the principles that have brought folks like us together, helping new people get into cybersecurity, learn where the resources are, know what it is, what the heck we're actually talking about. And a lot of things in cybersecurity are kept behind a facade of words and terminology that once you explain it, it totally makes sense. But it no one does a great job of explaining it. And I know a lot of great people who wanna help, And so I did the the little bit of organizing required to help put all those people in one place, or as many of those people as I could get at one time in one place.

Josh Mason:

And so we've got a bunch of training and lab companies, Hack the Box, SkillBit, Sans Institute, who will be there. We've got talk tracks that are all for new people. We've got workshops. I think we've got a security plus, like, crash course, how to study from CompTIA. So there's a lot happening in a little space in the community area of DefCon this year.

Josh Mason:

And then a bunch of other extra fun things happening too, because it's DEFCON and you gotta make it fun. But, yeah, the idea was there wasn't an on ramp at DEF CON for new people. A lot of villages have things, like, here and there. My friends at the Diana Initiative have been leading the charge of making it a welcoming spot for folks, but nothing glaring of, hey, you're new. We are here for you.

Josh Mason:

Like, ask all your questions. We will actively make a space to like onboard you, and then bring you into DEFCON, so that you gain value instead of just having a fun time, or being overwhelmed. Yeah.

Bronwen Aker:

Is it possible to go to DEF CON and not be overwhelmed?

Josh Mason:

I don't know. I always volunteer or run things when I'm at DEF CON. So So I've never actually been to DEFCON.

Ethan Robish:

So I think if I went for the first time, would you would a new new sec be a great village for me to

Josh Mason:

Yeah. Go visit? The new community would be perfect for you. It's for people who are new to DEFCON and people who are new to cyber security. Crazily, those are hugely overlapping Venn diagrams.

Josh Mason:

So there's a lot of folks who it's their first DEFCON who have never worked in cybersecurity before. So That

Bronwen Aker:

was me several years ago. I remember going to my first DEF CON and the Diana Initiative, they were just wonderfully warm welcoming people. And I had two posses that I was hanging out, the Women's Society of Cyberjutsu and also oh, I'm spacing on who the other organization was but you know, it's always nice seeing how much this community, the cybersecurity community reaches out to help people get in. Yeah, they're gatekeepers, but that's a whole other conversation.

Josh Mason:

Yep. And actually, I think I owe Mari and Roxy over at Cyberjutsu a conversation to see if they're if they have the capability. We've got a space available for them, and they joined us last year in the community in our the room that we had. So, yeah. Mary and Roxy are are great over there.

Josh Mason:

It was just they've been crazy busy. They just finished their own conference.

Bronwen Aker:

They did. And it I'm sure they're probably just face planting and and figuring out, okay, how do we do that again? So what do you do, Josh? I mean, I know what you do. I know, well, I know some of what you do, but why don't you tell us a little bit more about your background?

Bronwen Aker:

What do you do? What is your specialty? What what is it that keeps you awake at night, and what is it that gives you the biggest thrill when you're able to achieve it in your day to day?

Josh Mason:

So I'm kind of a weird animal in cybersecurity in that I, fifteen years ago, was a c one thirty pilot in the Air Force. And my job was to carry people and stuff into difficult places and then take them out when they needed that. And I I did really well. And then Air Force swapped me into cyber warfare, and I was essentially a deputy Sizzo in my first role there. Because I was an officer who had been in for a while, and I knew I learned all the lingo.

Josh Mason:

Learned all the tools. A lot of it I had known for a long time. But because my dad was a network engineer, and so I've just always been around computers. So it was like, oh, this is how you're supposed to do it. I went, oh, okay.

Josh Mason:

I've I've set up switches before, but never the right way. So now I know. And then I got to my job, and they were like, yeah, you're not allowed to touch switches. We have people for that. We need you to go to the meetings and talk to them about strategy, and then come back and let us do our job.

Josh Mason:

Well, okay. Excellent. And so I started in cyber leadership as my first job in cyber. It was a crash course and something that, like, I didn't necessarily ask for. I took a stint teaching at the Air Force Special Operations School, not teaching cyber, just teaching special operations stuff for a few years.

Josh Mason:

Taught by 50, like, workshops and courses to a few thousand students over the two years, and then got out of the Air Force and started teaching cyber up here in Maryland near Fort Meade, back to the to the military. Thought threat detection and pen testing, baby pen testing. They called it cyber threat emulation, but it's everything you would teach a baby pen tester. And then from there, someone introduced me to sales and sales engineering. It's a lot like teaching, but they pay you a lot more.

Josh Mason:

And no one has to pass the test at the end. They just have to decide. Yeah. So there's all these crazy tools. Some of them, like, I I'd love to chat about as we get into it, some of the topics that we discussed before.

Josh Mason:

How do you know if you want SentinelOne or CrowdStrike? You kinda wanna see what they look like and what the dashboards are like and how it might work in your environment before you spend, you know, upwards of, you know, $80,000 for your company to go and have this whole tool. Right? Well, someone has to give a demo, someone has to walk you through a POC, someone has to understand your business, and if the tool is a good fit. Salespeople are crazy at what they do.

Josh Mason:

A lot of the account execs and sales reps and wherever they call themselves that I've met over the years, They will remember people, they hold great conversations, they are the way less autistic people in cybersecurity. And then there's sales engineers.

Bronwen Aker:

Sorry. Like,

Josh Mason:

I know you have a dog. You said the name earlier. It's gone. But I think you use Starlink because technology sticks with me. And then you've got a Blue Yeti mic.

Josh Mason:

These are the things that, you know, stick to my autistic brain and You have your priorities. Yeah. When I get in a conversation with someone, if I'm when I was at Synapse, pen testing or at SimSpace selling training and ranges, knowing, okay, what do your people need? What do you need for the company? Like, what are you currently doing for pen testing?

Josh Mason:

How many sites do you have? I already know. I did my research. I can odysent and scan publicly. So I go into those like, how many do you think you have?

Josh Mason:

Because what I found was, I was in public sector at SYNACK, and so my clients, their bosses, you know, went to cabinet meetings at the White House. So they had thousands of domains, and hundreds of websites that fell under those domains. So you you well, Black Hills, but that's something that Black Hills, I don't know, has enough pen testers to handle. But SYNACK has like 1,500 pen testers. And a system for managing all of that, and that works really well for enterprise level that size.

Josh Mason:

We I know a lot of pen testing firms, a lot of great people. But doing something at a scale of that size is just crazy in general. So knowing

Bronwen Aker:

can't imagine wrangling that many pen testers. I just can't. I mean, it's it's worse than herding cats on a good day. Or Am I wrong,

Josh Mason:

Ethan? Managers who would like place people and then the yeah. The overall folks hiring and managing the pentesters and running that whole community. They did a great job there. Unfortunately, my position got eliminated selling into the federal government.

Josh Mason:

These past two years has been tough and choices had to be made, and I get it. So I'm currently in between, but I also run Noob Village, a nonprofit that works at DEFCON. I also run my own vCISO consulting company, Mason SC, Mason Security Consulting, because I'm not great at creating names. But not that sort of creative. So we talked about this a

Ethan Robish:

little bit earlier, but do you wanna tell us a little about what is a v sizzle?

Josh Mason:

Yeah. It's a a kind of unique animal. I keep using that term. I don't usually use that one. What?

Josh Mason:

Unique animal?

Bronwen Aker:

Yeah. Are are are you having an AI tell moment?

Josh Mason:

I gotta break myself out of my own hallucination.

Ethan Robish:

If if the if the top keeps spinning, then you

Josh Mason:

know it's a it's a dream. Yeah. Exactly. No. VSizzo is something that someone introduced to me.

Josh Mason:

It's consulting. There are companies that, just like they need pen testing, and they might not be able to hire it internally, they might not even have security leadership at all. And if you're a 20 person firm and half the company is developers, and then you got an HR person, and a mark few marketing people, and then some customer support, who is handling knowing all the things about cybersecurity and making sure that, you know, you're you are secure. But you can go out and hire a whole firm that can just handle everything. That's a MSSP, managed security services provider.

Josh Mason:

And knowing which one to pick is also difficult to choose. If you're big enough, there are big firms like and Ernst Young and Hamilton and Accenture who will handle all of those things for you. If you're not that big, you don't have that much funding, there are people like me who go, oh, you build software and people are going to ask you for a SOC two report. They go, what is that? I that explain all the things behind it.

Josh Mason:

You are taking in their information. You're gonna store it, and use it, and maybe have it interact with AI, and they're gonna wanna know what tools you're using and how you're protecting it and all of that. And so we're gonna need to bring in some auditors who will validate all of the things that we're doing. They go, well, what if we're not doing those things? Then we start a plan to start doing those things.

Josh Mason:

And they go, how do I do that? I go, I will tell you. I will walk you through it. What are you doing to make sure that, you know, you don't have malware on your the machines? And they go, I don't know.

Josh Mason:

Do we have malware on our machines? I go, okay. Let me find you some solutions. Like, what sort of computers are we working with? What environment are you in?

Josh Mason:

Are you in Microsoft or Google Suite or something creative that you all made yourself or a combo or and then you go from there. And having known what works and talked to a lot of people and had other clients and having my own mentors who I can go to and be like, these guys are in Google Suite. They're all on Macs, and they're all remote and v they're not even VPN ing in because there's nothing to VPN to. They go, oh, well, yeah. Who I've used in the past is maybe this or that or that.

Josh Mason:

And what's their budget look like? How involved do they need to be? Can they can you handle things on a retainer? Do they need someone else to triage all the alerts and manage all of that, and then, you know, hand it right back to them or hand it right back to you to explain? And I can do all that work.

Josh Mason:

And then if I deliver it to them, like, hey, here are the options. We could do it this way, this way, or this way. Here are the price points. Here are the reasons why I recommend this. But if you wanted to go with that, what are your feelings?

Josh Mason:

Hearing, you know, their mindset on those things, and then being able to help make a decision. That's what Aviso does. Right now, I'm finishing up incident response plans for a client, And because we they started down the GRC track themselves. They bill software for health care practitioners, and so they're gonna be under HIPAA. They knew that they needed certain security things, and so they started down a path.

Josh Mason:

They did good work. I came in, validated what they had already set up, found the rest of the requirements, used a couple, you know, platforms that help manage GRC that are great ways of plugging in your, like, NIST framework or HIPAA rules or whatever it is that you're working on. He saw the CEO I was working with, he saw all those things and could figure out a lot of them. Smart guy. But he also was running a company and was one of the primary software developers for the company.

Josh Mason:

So like, where is he gonna spend his time? He realized it's worthwhile just pay Josh like some money and I will handle this. Come back to him, say, hey, you have access to your cloud environment. Let's hop in there. Let's check these things.

Josh Mason:

It was like, awesome. This went really fast. If I had to go through that myself, it'd be a pain. Okay. But here's the other thing.

Josh Mason:

You need a pen test. You need a tabletop. And then we should probably go from there because there's a few other things that are required, and we don't want you to have a breach. We don't want you to have a breach and have HIPAA go out because there's fines from HHS that are bad. Yeah.

Josh Mason:

I agree. So now we're starting another path. We did the tabletop, learned some things, built the incident response plans, getting those back to them. There's some tuning for EDR. There's maybe MDR down the line.

Josh Mason:

A few things like that. And, yeah, that's kinda what I do.

Ethan Robish:

And you're really yeah. You really gotta know your stuff. I mean, I can see where your background in not only cyber leadership, but like, I imagine the sales engineering aspect helps with that too, and reaching for all these different solutions and like, oh, yeah. I have experience with this. And so I am I am kinda curious, given this is the AI Security Podcast, like, how you've been thinking about AI?

Ethan Robish:

How have you been using it, reaching for it to, I I guess, help you in your current endeavors? Or I don't know. May maybe maybe if wanna pivot and talk about other things, that's fine too.

Josh Mason:

I'm not ashamed to share a lot of the ideation I come up with. I will run through Claude. Like, hey, I've been researching this and this and this. I found these tools and they're these. Am I missing anything?

Josh Mason:

Is there anything new that came out that I didn't know about? And I can put it on research mode, and it's gonna go and find, like, hundreds of sources. I go and check Gartner and Forrester and all the info sec websites and do all the, you know, Google searching that I would be doing, but it'll do in the background while I'm, you know, doing other things. And then give me out like a little report, and I can go through there and be like, okay, this is this validates. It can look for reviews for, you know, what other people in similar issues have have chosen, what the industry says, what blogs have said, things that would take me a really long time to aggregate, and it does it very quickly with links.

Josh Mason:

So it's what I always hoped Google search would become in that way.

Bronwen Aker:

See, I use perplexity for that sort of thing, because Oh yeah. Perplexity not being an LLM, but being an answer agent agent first, I worry less about hallucinations with Purple Black City than I do with other LLMs, but you know. I

Ethan Robish:

get that. Yeah. Yeah. Was yeah. Because I was as you're describing it, I mean, I do the same thing for I mean, I'm not consulting, but, like, you know, my own research and things that I need answers to.

Ethan Robish:

And to Bronwen's point about hallucinations, like, I I always worry about being, you know, like those lawyers you hear in the news that go to a court with just hallucinated case law. And it's like, oh, they I said, like, so how how do you think about, like, confirming what it comes back with? I mean, obviously, you've got enough background experience to, like, sniff out, like, oh, this sound this sounds way bogus. Like, I'm gonna go check into this. But

Josh Mason:

Well, that's what I like about what I figured out how to use Claude. And Gemini does deep research in the same way. If it tells you a thing, it's going to have a link to where I found it. And so if it sounds interesting, then I'll go to the source and go read it up on that there. These incident response plans that I am coming up with.

Josh Mason:

I've had a form that I've used in the past, but I got curious. And I was like, I know CISA has IRPs out there. And the Australian, like, cyber defense agency. That's not their correct name. But essentially, their CISA also puts out IRPs.

Josh Mason:

And so I said, hey, go find me all the ones that might fit for these guys. I already had in my mind what I wanted to, you know, start with. Rants more hip hop, insider threat, that sort of thing. But I have a list. But, you know, am I missing anything else?

Josh Mason:

And are there things that are missing in my IRPs, in my draft, like, my templates that other people already have? And it was able to come back with suggestions and links. I went, oh, okay. Excellent. Can you draft me a version that includes all that?

Josh Mason:

And it, being one of those engines I can build, it, you know, built a first draft. Yeah. You know? There were things where I was like, you got creative here. Don't do that.

Josh Mason:

I Like, I know what you were trying to do. Don't do that.

Ethan Robish:

I love that use case. Yeah. And I I mean, I've done similar things to that before. Like, you take kind of the best of breed or the, you know, different things that have different aspects that you wanna pull from, and you kinda guide it. And but but the in that case, like, you're not having to come up with things completely from scratch.

Ethan Robish:

You're just having it, like, speed up the research that you might do yourself anyway.

Josh Mason:

Yeah. And I'm blown away by the people who are happy to let it just do thinking for them and create stuff. Yeah. Yeah. It's those people live in a world of a lot of trust.

Bronwen Aker:

Yeah. And Well, that or or they fall into the category I usually call sleepwalkers, but

Josh Mason:

Possibly. Well,

Bronwen Aker:

okay, so I have two questions for you Josh. Is, how much attention are is our small businesses paying to their security around AI.

Josh Mason:

Yeah, and that is an interesting piece. Even bigger companies. Both my clients, my friends, companies, and companies I've been with over the past couple years have had a lot of different views of using AI. And early on we knew if you fed it something, someone else might gain access to that. Apparently guardrails were put in place, and that shouldn't happen.

Josh Mason:

And now there's tenants, and if you have enterprise licensing everything should live within your tenant. Same way it does like a Google Suite tenant or a Microsoft tenant. I've interviewed red teamers at Synack who found out you can breach tenants because they were doing pen testing on AI models for clients. And that's what, you know, they attempted to do. I've chatted with my buddy Jason Haddix, and he's worked with a lot of companies over the years to find those those flaws.

Josh Mason:

And he keeps glasses on them. There are now people starting to worry, how do we govern? And if you're gonna govern, how do you enforce how people are using AI. Being on a VPN and having all of your traffic routed so that you can't go to certain domains is kind of the the bare bones way of trying to manage that. Doesn't really work that well because people will go on their phones or they'll go on another device or they'll turn off the VPN.

Josh Mason:

I know there are companies I've met, Marrow, m a r o, and they build a extension for your browser and a dashboard and a management platform, so that if someone goes to like Claude or ChatGPT, it'll detect that and then see what they're adding files or information chat typed in there, and then stop them and highlight, hey, this is a breach policy, don't do that. It's like DLP, but you know, browser based. There are some that are trying to work on the desktop. I know a lot of EDR companies and DLP companies are working on trying to include that in in their solution. The irony is they're using AI to try to do that in their solution.

Josh Mason:

Which then gets to the the really scary part. Everyone is using AI in their software. In this incident response plan process, I had to get with my client and say, hey, who is your cyber insurance and what do they require? Because I don't wanna build you this plan, you follow it, and then they go, yeah, you did all these steps, and because you did them in the incorrect way from our policy, we're not gonna, you know, pay for this. And they went, oh, well, we don't have insurance.

Josh Mason:

I went, okay. Cool. Let me add that to what we're doing. Went through the process, and in there, people don't wanna do cyber insurance without E and O, errors and omissions insurance. A lot of professionals have E and O insurance.

Josh Mason:

I know a lot of pen testers who have E and O, and the consulting world, I've got E and O. If I hand someone something and they sue me for $2,000,000, like, I'm done. I'm

Bronwen Aker:

done Yeah.

Josh Mason:

For that. Just because I put something in a report and they followed it. So you have errors and omissions insurance, just in case. And if they're a software company where people are going to use their tool and it's actively using AI, how is their tool going to avoid hallucinating? How is the ML or AI that they're utilizing in their agent going to be safe and avoid creating errors or omissions?

Josh Mason:

And a lot of insurers are underwriters are are looking at that. And so I had to get with them and say, what are your guardrails? Like, what are you using for tooling? And there's ways of writing agents and writing some of your tools so that they will only pull from the information given. They so they shouldn't hallucinate.

Josh Mason:

There are some tools that you can add on the back end if you're going through like AWS or Azure or GCP to access the AI for your tool. If you're developing on those platforms, you can use the AI through those platforms, pick your flavor, and then also put the guardrails around it with their tooling or third party tooling. And it's something that I didn't fully understand until I got, you know, elbows deep into it with my client.

Bronwen Aker:

It's Are you talking about rag sex or something else?

Josh Mason:

Oh, yeah. Yeah. I I've heard that one. That's not the one

Bronwen Aker:

That wasn't what you had in mind?

Josh Mason:

Well, that's retrieval.

Bronwen Aker:

Retrieval Augmented Generation.

Ethan Robish:

Yeah. Yeah.

Josh Mason:

Yep. And that's the baseline is having it be based off of what what you've got access to, rather than hallucinating and using the model, not necessarily the whole platform to access just the information given to it. Because you can train the model in any which way, but the underlying, like, root model is just a way of interacting with information, and it'll be based off of what it's been trained on. So if you only train it on certain information, and it's retrieval, augmented, only has access to those things, it should prevent hallucinations.

Bronwen Aker:

Does In theory. It does reduce the incidence. Though believe me, a determined LLM will hallucinate, You know, you just can't stop them.

Josh Mason:

Yeah. The so the is also big, is Yes. These things

Bronwen Aker:

back in this way. Fair enough so. So my second question was getting back to the fact that you've kind of specialized with the small businesses, and of course, we are an AI podcast. How do you see AI reshaping what small businesses do around their own cybersecurity? Do you see it having an impact?

Josh Mason:

Yeah. There is a lot more space for folks to come if, you know, they knew the questions to ask, they could like replace me within AI. There's a lot of additional work of fixing what the AI provides you, but they could probably get by with a good tool. And it's opened up kind of that window for people to learn a lot of information quickly that they would have had the time or capabilities of before. And I do see it a lot.

Josh Mason:

I actually had two opportunities for VisaZone that fall through because they were like, oh, no. We kinda got it figured out, and I think we're good. And I went, okay. If you do get overwhelmed, I'm here. So I'm I'm fine with that.

Josh Mason:

If you remove the, like, the barrier to doing something well, that then gives us all more time to do something new even better than before. And so if there's clients out there who feel happy with, you know, what they're able to find. If MSSPs or, you know, different security tooling is able to sell directly better, awesome. Then I'm going to find other ways to specialize and become more helpful. Because if they can do that, I don't have to worry about that one.

Josh Mason:

I can worry about the more difficult problems of, okay, how are we gonna like, what governance do you wanna use, and how are we gonna go about it? So it is making security a little easier for some people. In the same way that when you made a router have a dashboard that anyone could access, and they could you could set up security rules in there, It made it easier for the people at home. As opposed to, no, you've gotta get this sort of router and you need to install your own firewall. When you bake it in and have it be semi useful, it means that we can spend more time on more important things.

Bronwen Aker:

Like playing with dogs.

Ethan Robish:

Or playing D and D.

Bronwen Aker:

Playing D and D. We have to get a D and D game together at Walvis Hack and Fest.

Josh Mason:

I at Mile High, I ran three sessions of one shots, and at Deadwood, I ran four. So I plan on bringing I've got, like, big bags of dice behind my head over here, and I'll be taking those to hacker summer camp. People will be at parties and I'll be like, hey, who wants to go over here and play some D and D? I hope that sounds awesome. Were

Ethan Robish:

you were you at, what, way west this year? No. Mile high this year? Yeah. Yeah.

Ethan Robish:

Shoot. I I was there. I I missed out.

Josh Mason:

It's all good. Yeah. That's one of the things people are starting to get to know me by is all a big bag of dice from Amazon for, like, $20. And then has anyone played before? No?

Josh Mason:

Okay. Well, here. Now you've got dice, so at least you don't have that barrier. And some people are like, oh, I love to play. What color dice do you have?

Josh Mason:

And I'm happy to give them away. Other people have stickers. I've got stickers too, but, you know, it's nice to have a little thing to give away to folks at conferences. And a set of, you know, seven polyhedral dice is always nice.

Bronwen Aker:

Oh, you're giving them a full set. You're not just giving them like a d 20. No. Very cool. Very cool.

Josh Mason:

Oh, yeah. I've got I've got white and blue and red and orange, all the different black hills and anti siphon and colored d twenties. But no, the full set, you know. That way, so we can decide if they wanna be a rogue or a barbarian and you have the dice ready to go.

Bronwen Aker:

Well, Ethan, I feel bad. I've kind of dominated this guest interview with Josh because, of course, Josh and I know each other. Did you have more questions that you wanted to balance off of, Josh, before we wrap up? Because we've already got a half an hour so far.

Ethan Robish:

Well, it's a it's a couple things. I'm gonna go back to before I derail the D and D, and then I

Josh Mason:

can derail the D and D.

Ethan Robish:

So you're you're talking about, like, you know, that some of the opportunities you lost because presumably the client was like, oh, hey. Now we're chatting with Claude or whatever, which which is great. But I I think probably the way that it's going to mature and be sustainable is people like you or with the expertise that you have building those systems. Like like not just, oh, here's generic cloud, like, write a security plan. Like, it comes back and they have something, but they don't have the expertise to know what parts are BS or not.

Ethan Robish:

And if if you build some if you had, like, the guardrails around it, the checks and balances on the output, you wouldn't necessarily have to be the, you know, white glove consulting with them, but they would still get the same kind of like output. And you'd you'd have, I guess, a value add, a product kind of thing to I mean, any number of companies trying to customize AI to their own use cases. Right? But

Josh Mason:

So there are some sales techniques that I picked up over the years. There's a lot of great books, even things from Dale Carnegie that aren't sales specific still apply. And so an opportunity loss doesn't mean like gone forever. The great thing is, if you know, like, the value of the product and you build the relationship and the trust there, but they can't get over a hurdle of this is gonna cost even this much, and I'm willing to take the pain of trying to figure it out myself with this tool that's cheaper. If you leave them with, you know, that trust and that relationship, that, oh, well, if you're gonna go that route, could I suggest this?

Josh Mason:

And here's this as well. They leave going, oh, wow. He we're not gonna pay him, and he still wants to help out. There's a good chance they're gonna get overwhelmed. They reached out to you in the first place for something.

Josh Mason:

If they get the funding that comes through, if they make the revenue that they need, then they remember you. I've actually had a couple clients who were lost opportunities that six months later they went, okay, we wanna do this now. Like, the contest that we did, that we found the people and we handled it, we weren't happy with the results. And I go, I know. I know that for me.

Josh Mason:

That's not who I would have suggested. Because I've been in this. And you know the people. You know who does the work and turns out good reports, if you've been around for a while. Same goes for, okay, we we were looking at this product, and we didn't like the dashboard at all.

Josh Mason:

I go, I know. I hate that dashboard. If they feel the pain, they'll come back if you leave them liking you. And frankly, you can't get over certain mind hurdles. If someone has, like, decided that they just can't pay for that right now, that consultant side of it, and the crazy thing is, if I find them like a tool or I find them a pen tester, it doesn't cost them anything.

Josh Mason:

Usually, you get referrals from, you know, the vendor. So like, I would get paid from, you know, the pen testing firm, you know, a little bit here, a little bit there for bringing someone in. And so it's one of those of it doesn't hurt them at all, just trust me. But they find out through, you know, time that it's like, oh, well, okay, then yeah yeah yeah, you come fix this next time. And I've had clients for some of my longest clients.

Josh Mason:

We started that way. They're like, oh man, I don't know if I could afford this, or I don't know, I'm gonna go with this one. I'm like, okay. But here, I'm gonna send you this. And I'm gonna like, I follow-up later and like, hey, how how did that go?

Josh Mason:

Like, how is this going? And it sales is wild. Yeah. It

Bronwen Aker:

makes a big difference. I mean, I remember when around y two k, I was doing a lot of ecommerce training and a lot of people would come in and again, this is when the web is new and people a lot of the same stories you're hearing around AI were going around about web development, Oh, just build a website and you'll get rich overnight. Yeah, right. Well, they'd come into one of my classes to learn how to do basic web development and they'd go off and two or three weeks later, I'd get a call or an email saying, This is harder than I expected. But again, what you just said, you reach people, you offer them assistance, obviously within limits if they're not paying, but still you want to leave that positive relationship there.

Bronwen Aker:

And that's what I really hope is going to happen with AI is it's going to allow us to get past the drudgery of all the work that needs to be done for all these systems we've created and it'll allow us to get back to having that connection with other humans.

Ethan Robish:

I like that.

Josh Mason:

Yeah. Sounds like a good

Bronwen Aker:

place to wrap?

Ethan Robish:

Yeah. Positive worldview. Twice in one week, who

Bronwen Aker:

is this friend friendly, warm, fuzzy Bronwen? So Josh, if people want to get in touch with you, how do they reach you? What is your domain? Or do you have any upcoming events that you're gonna be attending that you want people to come up and meet you at?

Josh Mason:

Mhmm. So I will be at Hacker summer camp. Monday evening at 5PM is my talk on careers with Jon Stoner at b sides Las Vegas. And then Wednesday and Thursday, I will be all over Mandalay Bay for Black Hat. I'll be introducing speakers.

Josh Mason:

So I might be busy. You might see me introduce the speaker.

Bronwen Aker:

There's a Josh sighting. And then

Josh Mason:

I'll be at the Noob community all weekend long for Defcon. People can reach out to me on LinkedIn. I'm linkedin.com/in. Joshua c Mason. I'm Joshua Mason with the mushroom behind my name.

Josh Mason:

That's a whole other story. I'm back from my flying days, and I'm also a fungus cyber penguin on Instagram and TikTok, I think. I should I'm supposed to do a TikTok at some point, my my friend told me.

Bronwen Aker:

That's how I would get little

Josh Mason:

little learn cyber. Yeah. On the TikTok. So, yeah, those are the best places to find me. Noobvillage.org, noobvillage.org, has a link to our Discord.

Josh Mason:

Anyone can join, and I'm right there. Most of my posts are, hey, we've got a staff meeting today or on Thursday. And then I run a open staff meeting on Thursday nights at least every week for July leading up to DEFCON. And people are I don't know. Some people find me on there and chat.

Josh Mason:

Or if you need consulting, mason dash s c dot com, because the other Mason SC goes to some town in South Carolina. But mason-sc.com is my domain for all my consulted stuff. Got some good resources in there on the blog and yeah.

Bronwen Aker:

Cool. Thank you so much.

Ethan Robish:

Meeting you and get getting to know you a little bit. So thank you very much.

Josh Mason:

Thanks, Ethan. Thanks, Bronwen. It was great chatting with y'all.

Bronwen Aker:

Alright. So with that, thank you once again for joining us at AI Security Ops and keep prompting.