AI Security Ops

In this episode of AI Security Ops, the team tackles one of the most common questions security teams are asking about open-weight AI models:

Are foreign open-weight models actually a security risk?

Not in the vague “AI is scary” sense. Not in the headline-driven “it must be spyware” sense. But in the practical, security-operations sense: if you download a model like Qwen or DeepSeek and run it locally, what risks are real, which ones are overblown, and what should defenders actually care about?

The answer is more nuanced than “ban them” or “they’re totally fine.”

Open-weight models can be cheap, capable, and private when they run on your own hardware. But “open-weight” does not mean “open source,” and running a foreign model locally does not automatically mean it is phoning home. The bigger risks are often in the runtime, file format, download source, tooling chain, model behavior, and how much trust you place in the output.

We dig into:
- What “open-weight” actually means, and why it is not the same as open source
- Why the “phone home” fear is usually the wrong threat model for local weights
- The difference between a hosted AI service and a locally run model
- Why model delivery, runtime, and tooling matter more than the weights themselves
- How pickle files, unsafe formats, and poisoned packages create real supply-chain risk
- Why typosquatting and fake model repos are a practical concern
- Why safetensors and verified sources matter
- How bias and censorship can show up in foreign and domestic models
- Why model behavior, refusals, and blind spots can become integrity risks
- What sleeper-agent research tells us about hidden triggers and model backdoors
- Why country of origin matters, but does not replace basic security hygiene
- How to safely evaluate and use open-weight models in real workflows

This episode explores a critical shift in AI security: the risk is not just where a model comes from. It is how you download it, how you run it, what data it can access, what actions it can take, and whether your pipeline assumes the output is trustworthy.

For security teams, the practical takeaway is simple: do not treat any model as inherently safe just because it runs locally, and do not treat every foreign model as magic spyware. Build the workflow so the model can be useful without becoming a single point of trust.

—

Key Concepts & Topics

Open-Weight Models
- Local model weights and inference engines
- Open-weight versus open source
- Qwen, DeepSeek, and foreign model adoption

Threat Modeling
- Local models versus hosted AI services
- The difference between weights, wrappers, and APIs
- Why “phoning home” is usually a runtime or tooling issue

Supply-Chain Risk
- Unsafe model formats
- Pickle files and arbitrary code execution
- Typosquatting and poisoned repositories
- Package and dependency compromise

Safer Model Handling
- Prefer safetensors over risky serialized formats
- Download from verified sources
- Pin hashes and validate model artifacts
- Use containers and restrict unnecessary network access

Bias and Censorship
- Model behavior shaped by training data
- Political, cultural, and regulatory influence
- Refusals, blind spots, and subtle output bias
- Matching model behavior to the use case

Sleeper Agents and Backdoors
- Hidden trigger behavior in model outputs
- Why behavioral testing may miss certain risks
- The difference between lab demonstrations and real-world evidence
- Designing workflows so hidden triggers have limited impact

Defensive Strategy
- Treat model output as untrusted input
- Do not pipe outputs directly into shells, databases, or production systems
- Avoid unsupervised code execution or autonomous production access
- Make adoption decisions based on threat model, compliance, and use case

Learn more about Black Hills Information Security:
https://www.blackhillsinfosec.com/

Check out Antisyphon Training:
https://www.antisyphontraining.com/

#AISecurity #CyberSecurity #LLMSecurity #ArtificialIntelligence #InfoSec #BHIS #Antisyphon #OpenWeightModels #SupplyChainSecurity

  • (00:00) - Intro: Foreign Open-Weight Models and Security Risk
  • (01:50) - What Open-Weight Actually Means
  • (03:35) - The Phone Home Concern
  • (07:44) - Pickle Files and Supply-Chain Risk
  • (14:34) - Bias, Censorship, and Model Behavior
  • (19:21) - Sleeper Agents and Hidden Triggers
  • (24:41) - Country of Origin vs Security Practices
  • (25:20) - Practical Checklist and Final Takeaways

Click here to watch this episode on YouTube.


Brought to you by:
Black Hills Information Security 
https://www.blackhillsinfosec.com

☯️ Introducing BHIS Fusion Penetration Testing
https://www.blackhillsinfosec.com/fusion-penetration-testing/

Antisyphon Training
https://www.antisyphontraining.com/

Active Countermeasures
https://www.activecountermeasures.com

Wild West Hackin Fest
https://wildwesthackinfest.com

🔗 Register for FREE Infosec Webcasts, Anti-casts & Summits
https://poweredbybhis.com


Creators and Guests

Host
Brian Fehrman
Brian Fehrman is a long-time BHIS Security Researcher and Consultant with extensive academic credentials and industry certifications who specializes in AI, hardware hacking, and red teaming, and outside of work is an avid Brazilian Jiu-Jitsu practitioner, big-game hunter, and home-improvement enthusiast.
Host
Derek Banks
Derek is a BHIS Security Consultant, Penetration Tester, and Red Teamer with advanced degrees, industry certifications, and broad experience across forensics, incident response, monitoring, and offensive security, who enjoys learning from colleagues, helping clients improve their security, and spending his free time with family, fitness, and playing bass guitar.
Guest
Ethan Robish
Ethan Robish has worked with Black Hills Information Security (BHIS) since 2008 — first as an intern and then as a full-time Security Consultant starting in 2012. In his current role as a Threat Hunter, Ethan is involved with customer engagement, research, working with Active Countermeasures’ AC-Hunter, as well as improving BHIS HTOC and SOC offerings. Previously, he implemented defensive security solutions for the Exchange Online security team as a Microsoft intern. While in college, he competed in the International Collegiate Programming Competition (ICPC) World Finals. In his time off, he enjoys cooking, playing the piano, and reading fantasy novels.

What is AI Security Ops?

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

Brian Fehrman:

Hey, everyone. Welcome to this week's episode of AI Security Ops. Openweight models from foreign labs are everywhere now. Gwen from Alibaba, DeepSeek, and others routinely are topping the open leaderboards. They're free to download, and they run entirely on their own hardware, potentially.

Brian Fehrman:

Developers love them because they're cheap, capable, and private in the sense that nothing leaves your machine. But made in China, runs on my laptop, raises questions for some security folks. Are we inviting a risk that we don't fully understand? And today, we're gonna separate out the real threats from the hype. Spoiler, the answer isn't they're malware, there's spyware, and it isn't that they're totally fine either.

Brian Fehrman:

It's more interesting than both of that. So before we kick off, we'll let you know about Black Hills information security. If you or your organization are in need of any security services, whether that would be external, internal testing, soon compromised, cloud tests, wireless tests, physical security, SOC services. Anything and everything that you can think of security related, we can help you out. Check us out at blackhillsinfosec.com.

Brian Fehrman:

We also have a training branch, antisiphontraining.com, where many of our consultants who are doing this stuff day in and day out, they take up what what they have learned, package that knowledge into an easy to digest and affordable format for you to consume and hopefully, you know, level up your day to day, job skills, help you get that job that you're looking for, or maybe just further along in your hobbies and interests. So with that, let's go ahead and let's kick it off. Let's talk about, the first one of the first concerns that people have, which is the phone home concern. What do you guys think about this one?

Derek Banks:

Yeah. So first, just a quick frame, what a open weight model is. It means you get the model's parameters and then run it yourself through what's typically called an inference engine. There's lots of different ones out. You've probably already heard of them, like ollama, llama dot cpp, vllm.

Derek Banks:

There's MLX on the Mac. There's lots of different inference engines. And just as you might suspect, people have their opinions and preferences about such. But open weight doesn't mean open source. Usually, you don't get the training data or the full code that was used to train the model, and that distinction kind of matters.

Derek Banks:

Right? And even if you did have the training data and the code to make the model, chances are you don't have 5,000 h 200 sitting around to, run the the training run. And that's the thing. And I recently, Brian and all, were on a sales call where the a vendor was claiming that they trained their own LLMs, and I still think they don't know what they're talking about. I mean, I mean, maybe they do have, you know, a few thousand, you know, $40,000 video cards or GPUs, but I doubt it.

Derek Banks:

They're probably doing supervised fine tuning or something else that they're calling training that's not really training. But anyway, I digress. It means that you basically get a black box that has magic math in it. That's the open weight model.

Ethan Robish:

It's kinda like the term freeware for for software. So you can have open source software, or you can have freeware. Both of them, you can download and run for free, but one of them has the source code available, and the other doesn't.

Derek Banks:

Yeah. It's a good parallel. And so, you know, the phone home thing, I think, is kinda misplaced. So the scenario is is that I'm using some kind of, agent to interact with, the open weight model. The popular ones now are, what, like Hermes, OpenCode, I think Codex and Claude, with Olama.

Derek Banks:

They have, like, some kind of wrapper code where you can run Claude code and her and and Codex with, you know, open late models. And so the the the fear here is is that you're setting it off to do some kind of coding task or something for you, and silently in the background, it is coding a backdoor that's going to exfiltrate your data or somehow compromise your machine. And, to me, I think this is the least credible threat. I won't say it's out of the realm of possibility because here in 2026, I don't know. It seems like a lot of what we're doing is borderline magic in a lot of ways.

Derek Banks:

I still think that if I took my MacBook Pro back to 1985 with an open weight model and showed people in 1985 what, you know, a quen three six could do, I'd probably be burned at the stake as a witch. Right? Like, it's it's kind of cool and magic and and amazing. But I think that, you know, a couple of things. One, getting that in the training data where it would deterministically pick, like, a place to send the the the exfiltrated data or to phone home.

Derek Banks:

Like, that seems like that would be kinda tricky. Maybe it would be hit or miss. But the other thing is is that, I mean, these are companies that are putting the stuff out for global, you know, distribution so that eventually they can make money. I mean, even the Chinese, that's what they want to do is make money and saturate the market, maybe undercut, you know, US competition. If they had some kind of, you know, I'll say backdoor, but we'll use the term backdoor later to mean something a little bit different.

Derek Banks:

If they, you know, trained in the ability for it to code a a a backdoor to exfiltrate data, I feel like it would get caught pretty quick, and then a company like, say, z I z AI or Alibaba or whatever, their future models would be blackballed, at least in my opinion. Like, no one would use them again. So I wanna say it's out of the realm of possibility, but I don't really think it's a concern.

Brian Fehrman:

Yeah. I think it's I think it's extremely unlikely, especially I mean, we're talking about the the model itself. I mean, sure, maybe if you pick something up that has that's like a wrapper around a model, an application that's that's put out, sure. There could be more concerns there. You start getting third parties involved.

Brian Fehrman:

There could be potential concerns there. But when we're talking about just the model itself, it's

Derek Banks:

You're right. I wanna make the distinction. I am talking about if you go to a reputable source and you download the collection of weights that is the model file, the 36 gig or whatever it takes for the that'll run on your laptop, and not using the API service in China for the model. Right? Not that.

Derek Banks:

Using it locally in your machine, you went out to Hugging Face, and you grabbed the model, and you're running it now. That is what I'm talking about is that I think that this, you know, phoning home thing, I just don't think it's practical, and I would be amazed if someone were able to training in and get it to work, like, consistently in practice.

Brian Fehrman:

Yep. Yeah. I I agree. So I guess I can move us on to the to the next one here of when the models themselves could potentially be weaponized, but not within the weights necessarily, but more potentially in how the model has stored itself. In particular, there are so there are different ways that you can distribute a fully trained model, especially with these open weight models.

Brian Fehrman:

There's more than one format just like there's different image formats. There are different model model formats, basically. And one of them in particular is what's known as the pickle format, and so that's where we can start getting into things like supply chain attacks, which have seemed to have been on the rise or at least in the news more prominently recently, in particular with, things such as a squatto or squat typo squatting. Or squatto typing. Her just

Derek Banks:

gonna I like squatto I think

Ethan Robish:

you just coined a new term. Yeah. It's squatto typing.

Derek Banks:

I feel like that's what Claude tells me when it's working on something. It's like, now we're squatto typing. Yes.

Brian Fehrman:

Yeah. Yeah. So, you know, someone someone gets access to, to a a repo that's out there or they do typo squatting, to, over to basically put their own weaponized version of the model in a dangerous format, such as a pickle format, then that that could be a real risk of potentially having code run on your system. But again, it's not it's not the open weight model itself. It's a weaponized version of the storage.

Brian Fehrman:

So what are but I think that this is largely mitigated at this point, isn't it? I have some better formats to store models in.

Ethan Robish:

Yeah. And and just to clarify, Pickle is like a Python library, and this particular issue has been is not specific to AI. Like, anything distributed as as like a a pickle serialized file has this risk where you deserialize untrusted pickle files, it's a risk of arbitrary code execution. So AI just open weights being distributed in this pickle format inherits this risk. And I think what you're saying, like, one of the things they've they've done to avoid this is just to stop using the pickle format.

Ethan Robish:

I think there's probably better for sure, safer formats that that can be used to to distribute these weights.

Brian Fehrman:

Yep. Yeah. Safe Oh, go ahead, Bear.

Derek Banks:

I was gonna say, I think probably one of the most popular one at the moment that works with llama c p p and and llama is g guff. And so you'll get the tokenizer and the open weight stuff, and then, and the open weight model, and then your harness will be able to decipher that.

Brian Fehrman:

Yep. Yep. And the safe safe cancer format is is Yeah. One big neither of those have the have those same, exerialization concerns that, Ethan Ethan spoke about.

Derek Banks:

It doesn't mean that they might not have concerns in the future, as more and more of the stuff is, you know, you know, used and and scrutinized, and but all you know, when your your file format typically, it seems like to me depends on the inference engine that you're choosing, and you'll typically choose an inference engine kind of based on, like, your requirements. And again, if you're just if you're listening to this and you're just getting started, probably the place to start is Olama, I think, or maybe Olama CPP. But Olama is a great place to start and, you know, it while it might run a little slower than if you're using Olama CPP or or VLM or something like that. I I think it's a lot easier to use, and I think if you're just the the neat thing in my opinion about Olama is you can just Olama space pull as a command and go get it, you know, for a trusted version from from Olama's, you

Brian Fehrman:

know,

Derek Banks:

repository.

Ethan Robish:

So have you guys run into this? I I know it's talking about, like, not just the the model itself, but, like, all the code that's surrounding it. Like, the the whole repo might have extra code that configures tokenizers or sometimes running Python code. Like, have you guys seen that?

Derek Banks:

I I personally have not, but it would not surprise me.

Ethan Robish:

Yeah. Mean, they do something similar with so, like, a lot of the supply chain attacks, like an an NPM package gets compromised. Like, NPM specifically, if you pull it down and install it, there's code that it will just automatically run. And that's that's where they were poisoning the packages is, like, in this autorun code versus if you're using like BUN to install the the program. I just Gotcha.

Ethan Robish:

The package. It just would ignore that. So I like you said, I I could see this happening.

Derek Banks:

Yeah. So I've definitely you know, I think supply chain risks are probably like the really our biggest problem at the moment. And so there's been a couple of different, you know, types of those, and so you're describing the poison packaging one. I think Light LLM, which was a really scary one happened a couple weeks ago. Maybe it was more than a month ago now.

Derek Banks:

I don't know. Teenagers all the time like blurs. I don't know. But that one was particularly scary because, you know, you could just be using claw goat. In fact, I think it got discovered this way where somebody, you know, all of a sudden noticed that their project was packing their CPU, and they went and looked and found out that there was a compromise, a light LLM package that got installed from the official repository.

Derek Banks:

And so and if I if I unless I'm crossing the streams, I think that what happened to Light LLM is they had, like, some flaw in a git runner or something that got compromised, like

Ethan Robish:

I think it was part of the Aqua Security Breach.

Derek Banks:

Maybe. Yeah. Irregardless, to me, that is not going away anytime soon. And again, like you were saying with Pickle, it's not like it was specifically, you know, like, this is a specifically an AI thing. Right?

Derek Banks:

Like, we've pointed out many many times, like, mean, how many of us have done PIP install as root on a server before? If y'all didn't raise your hands, I know you're not telling the truth. Right? And so, I mean, these things have been around, like, there've been, you know, beautiful soup is a good example of a misspelling that, you know, was weaponized because nobody could spell beautiful like the first time correctly. Right?

Derek Banks:

And so, you know, this kind of like squatotyping thing happens. And You're make it a thing. Yeah. And so, you know, the mitigation for this, there's like two things that I can think of that like would help. One would be to use a package manager like poetry or uv with your Python code to pen package versions to known good versions.

Derek Banks:

The other thing is you can use an age gate, which I think a little bit more complicated to pull off, but don't install any packages for Python or NPM that are, you know, younger than seven days or something like that. Those things should keep you a little safer for supply chains, but this this is gonna be, like, you know, the jelly of the month club. It's just a gift that's gonna keep on giving, in my opinion.

Ethan Robish:

Yeah. I think we agree, like, that's probably the biggest risk. That's why you're kinda focusing on it again. But but on onto the next potential risk from open weight models is this one actually rings a little true to me too, is built in bias and censorship. I mean, I would say every every person comes with their own biases.

Ethan Robish:

It makes sense that, like, models that are trained on output from humans also also have biases. And, you know, where they train, like, what makes up their training data just probably has some inherent biases. And maybe there's intentional biases put in too. I think famously, some of the Chinese open weight models refused to discuss certain Yeah. Yeah.

Ethan Robish:

Well Oh, you just you just got us flagged in China to be fair.

Derek Banks:

Oh, well. Sorry. Well, I'll pick on Google too. This happened with Google as well. Right?

Derek Banks:

When they released a model that, you know, there were people who were asking questions, and I'm not not being political or anything, but they were like asking questions about like the founding of The US, you know, and and show me pictures of, you know, the, make pictures of the founding fathers, and they would be, like, native American or something. And, like, well, that's not historically accurate. Right? And so what was happening is in their training dataset, like, as they were training this model, they were taking current, like, views and trying to put safety constraints around the model so it would do certain things. And I think the president of Google or CEO of Google at the time after this all came out that it was basically just not even putting out accurate information at all, said that they they kind of missed the goal with this model.

Derek Banks:

And, yeah, that that seems to be true. And so and then, know, if you again, you know, not to be political, but this is the big thing that Elon Musk is talking about is, like, if you train him bias, then it's a bad bad thing moving forward for the future. But honestly, like you said, Ethan, I mean, even if you don't intentionally do it, what you choose in your training set data is going to have bias. And you as the data scientists are supposed to be able I mean, this is what I learned in the data science masters is you have to be aware of the bias and test for it and and try and get it out of your model, in this case, an LLM. You know, like, this applies to any machine learning model is because it's all based on your dataset and the algorithm that you're using, and that there's bias in your dataset.

Derek Banks:

And even, you know, we say bias as like, you know, a person having an opinion. Bias also means like, we could have statistical bias, right, that you're not accounting for. And, you know, to me, it's just something that the the people who are making the model should be accounting for as they train and and tune them.

Brian Fehrman:

Yeah. Certainly. Yeah. It's been a problem since a long, in terms of AI. Bias has obviously been a problem for a very long time, but when we're talking about AI stuff, it's it's been there's been issues with it long before generative, AI was more popular with LLM.

Brian Fehrman:

So, like, classifiers, for instance, in particular, there are stories with facial recognition that was put out that work better on certain certain skin colors versus others. Yeah. That wasn't an intentional thing that that the people did if shit like, they just didn't think that.

Derek Banks:

They picked the wrong pictures. Right? Yeah. Picked the pictures and

Brian Fehrman:

realized, like, oh, hey. Yeah. We have a bunch of this skin color and our, and our training set. This is yeah. This obviously, this is gonna cause some issues.

Brian Fehrman:

Yeah.

Ethan Robish:

Do you think that was an a natural, like, bias that happened? Or I I kinda thought it was so what Derek was saying, like, the data scientists identified that, oh, there's there's biases, and there's, like, harmful biases of, you know, prejudice and racism. And they tried to, like, train that out, and it almost like overcorrected the other direction.

Derek Banks:

It could be that. It could be in any number of things. Actually, there's a really good book I read a couple years ago. I can't remember the author's name. She wrote a it was Kathy something maybe.

Derek Banks:

It's called Weapons of Math Destruction. And this is really before the LLM hype, and I was talking mainly about classifiers, and they were talking about things like with the prison system, with job hiring, like there's a lot of of interesting topics they brought about. It was all about bias being baked into into that. And I think I remember some I don't know if it was in that story, but Brian, what you're talking about, I I remember reading a story that the researchers didn't do on purpose. It's just a dataset.

Derek Banks:

Mhmm. Just went on the Internet and grabbed a dataset. Right? I mean, that's And that's what and not having the step in their process to do the evaluation to look at the data and make sure that it was a representative sample of, you know, in the bell curve of of what hands look like when you put it under the sensor or whatever. Right?

Derek Banks:

And it's it's actually kind of a hard problem, and it's something that, you know, the people who are making machine learning models need to account for. Alright. The last one is the sleeper agent or the backdoor question. And on this one

Ethan Robish:

I feel like this kinda comes around to what you were talking

Derek Banks:

about at the first point. Yeah. But I think that Collapse a little bit. This means a little bit different. So one of the things that we Brian and I have noticed in in the last, what, two is it been two years now, Brian, since we've been focused on AI?

Derek Banks:

I think it's been about that much. Oh, crap. That that they we we found that the folks who were talking about, like, AI red teaming, and and and what how you would, you know, red team an AI were really talking about the safety aspect of AI and what the model would output. And they use the term backdoor differently, like, than what we think about as a backdoor in computer security. They weren't talking about a backdoor where, you know, an external entity would get remote access to your computer.

Derek Banks:

They were talking about a backdoor where you could give a sequence of tokens and get a different output, like, a an unexpected output. And I think that's what backdoor means in this sleeper agent, example.

Brian Fehrman:

Yeah. I think so. I think that's what it's going to. So basically, like, if it sees certain keywords so the example it gives here, if it sees twenty twenty three, it might output, a certain type of code for a particular cast versus a besieged twenty twenty four, then that triggers a different behavior of in terms of in terms of output.

Derek Banks:

Although, Ethan, you are correct that they are related because it's all about the token stream coming out of the model.

Brian Fehrman:

Mhmm. Yeah. And so, I mean, there from a security standpoint, you know, there could be, two different risks there. One is so the generative aspect, in particular, if you're doing coding, tasks is for the ability for that output to be morphed based upon particular trigger output. But then also, if you have the ability to call tools, then it could be that it attempts to trigger a particular tool call.

Brian Fehrman:

If, you know, if this is agentic based implementation, based upon certain, you know, tokens tokens that it gets. But it seems like, again, that this is at this point anyways, more theoretical than a practical thing. It looks like there's been at least a little bit of research that Anthropic did.

Derek Banks:

Yeah. I just I feel like a lot of the security related stuff that come out of the lab sometimes are, I don't wanna say contrived, but they're constrained in a way for maximizing, like, the impact and getting the headline. Right? Mhmm. Like the the example we covered on the in folk the recent in focus episode after the news was an article, I think it was written by The Economist, where my state senator Mark Warner was quoted as mythos hacked the NSA's top secret computers in a matter of minutes.

Derek Banks:

And everybody who knows anything about classifiedcom computer systems and computer hacking basically went, yeah, tell me things that didn't happen for August, Alex.

Ethan Robish:

That's not

Derek Banks:

at all how that possibly worked. And it turned out when more information came out, yes, the scenario was, we'll just say, set up in such a way that that's what it made it look like. And so, you know, I I think this kind of thing is definitely possible. But again, I mean, the it's expensive to make these things. Right?

Derek Banks:

I mean, the the the even the open wave models, I know that I think China's oh, China. I don't I don't mean China. I think the company that made DeepSeek, claimed that they trained DeepSeek with, $6,000,000, And that claim, there's a company called Semi Analysis, basically called BS on that because there's there's just no way. Right? And so it's probably more like a couple of million dollars, you know, and, or a couple 100,000,000 actually.

Derek Banks:

And so, you know, I I think that again, Brian, I'll quote you, what does it take to train an l l LLM? Answer is money and a lot of money. Right? And, so I just I I I think I do think that there are definitely models that'll introduce, like, bias for the people who are making them. But I think the line is gonna be drawn of them having some kind of malicious, stuff that go on on your computers.

Derek Banks:

They they just they wouldn't survive as a company. Nope. I think they would be ousted pretty quickly. So I guess, maybe, you know, is do you think that our, US bias because anytime I've said, you know, where you you know, I'm using a Chinese open weight model, I kinda get the, really? Kinda thing.

Derek Banks:

Because I don't know that most people realize that the most of most of the capable open weight models, that are out right now are Chinese.

Ethan Robish:

Either Chinese or fine tunes of

Derek Banks:

Fine tunes of Chinese based models. Yeah. Exactly. And so, is it about the country, or is it the practices? Like, why why do people automatically think?

Derek Banks:

Think? Is Is it it because because the the Chinese Chinese have have spent spent decades, you know, hacking The US? And I mean, I'm sure we do the same thing. Right? And so do you think folks are in China going, man, I don't wanna use those those US models?

Ethan Robish:

It's funny because The US has, like, history of putting backdoors and things too. NSA backdooring cryptographic Yeah. Algorithms, putting harbor backdoors and things. It's yeah. It definitely, I would say, a country bias, like, where you live.

Derek Banks:

Yeah. So I guess we have a checklist of, you know, practical things that if you're going to run open weight models, and I I strongly suggest to that the audience listening do so. It is very if you're if you wanna get into AI, learning how to run open models and interact with them is a good way to start. And, so a couple of things, download only from verified official repos, pen hashes for those. Prefer safe tensors over pickle, and that was reminds me of Rick and Morty in the big pick Rick Rick episode.

Brian Fehrman:

I

Derek Banks:

like this one, run an inference in a sandbox or container, with no network egress if you don't need it. Although, I would also argue that for most people, you you do want to use the Internet with it. Right? Like, you want it to go out and use the Internet to get more information because, you know, once that model's trained, it doesn't know anything past its training date. And so if it needs to it'll need to go out and get more information.

Derek Banks:

But since you're letting it out on the Internet, that's why you should run it in a container.

Ethan Robish:

And and you could restrict it too. I mean, treat it like maybe you treat your your kids, like, rather than giving them unfettered internet access.

Derek Banks:

You only get to go to these sites.

Ethan Robish:

Yeah. Yeah. Net net

Brian Fehrman:

net nanny for AI. Yeah. I

Ethan Robish:

think net nanny is squatto typing.

Derek Banks:

So another good, thing on the checklist, treat model output as untrusted output. If you're coding together pipeline stuff, don't pipe it straight straight into a shell or into a database, and then match the model to the job. And and, like, if you're just experimenting, don't use customer data. I'm not saying you shouldn't use customer data with open weight models. Just be aware of what your, you know, your inputs and your outputs are.

Derek Banks:

And then all as always, especially if you're doing this at work, check with compliance.

Brian Fehrman:

Yep. Make sure you're would not need to be shadow IT ing it and yo on

Ethan Robish:

it. I I would be remiss if I didn't mention several of these points. So container sandboxing, we had a whole episode on different layers of security that you could put around your agent harness.

Derek Banks:

Yeah.

Ethan Robish:

And as far as using customer data, and Derek touched on at the beginning about, like, downloading these models and running them locally versus running the same model, but, like, on a hosted service. Like, you know, be aware of where you're sending your data, like data provenance. We've got a whole episode discussing that that kind of thing too.

Brian Fehrman:

Cool. Yeah. I think we covered a lot a lot of good ground. Anyone else have any closing thoughts? No.

Derek Banks:

I think we're ready to, give a conclusion and wrap it up.

Brian Fehrman:

Alright. Let's do it. Yeah. Well, I hope everyone, enjoyed this episode and learned a little bit. Hopefully, eased some of the concerns that some people might have about using, foreign models and put things into a bit more perspective to make people a bit more open and not so, not cringe up right away when they hear someone talking about using, using a foreign, open weight model.

Brian Fehrman:

So with that, we'll catch you on the next episode and keep on propping.