Red Team Blue Models

In this narrated edition of “Red Team, Blue Models: Adversarial Testing in an AI-First World,” we walk through what happens when “we did a red team” becomes a comforting slogan instead of real assurance. You’ll hear how AI-first systems create a new attack surface that extends far beyond jailbreak demos, into the messy intersection of models, prompts, plugins, data, and identity. We break down why adversarial testing has to focus on attacker goals, long-lived systems, and cross-layer behaviors if you want a risk picture that actually matches the stakes for your organization. This episode is based on my Wednesday “Headline” feature from Bare Metal Cyber Magazine.

From there, the discussion moves into the practical choices leaders have to make. We explore what a serious AI red team function looks like, how “blue models” can continuously simulate and detect abuse, and what it takes to integrate adversarial testing into AI delivery without strangling product velocity. You’ll get leadership-level patterns for governance, evidence, and metrics that go beyond counting jailbreaks and toward measuring real exposure. If you are signing off on AI features, building AI platforms, or advising boards on AI risk, this episode gives you language and mental models you can use in the rooms where those decisions get made.

What is Bare Metal Cyber?

Welcome to Bare Metal Cyber, the podcast that bridges cybersecurity and education in a way that’s engaging, informative, and practical. Hosted by Dr. Jason Edwards, a seasoned cybersecurity expert and educator, this weekly podcast brings to life the insights, tips, and stories from his widely-read LinkedIn articles. Each episode dives into pressing cybersecurity topics, real-world challenges, and actionable advice to empower professionals, educators, and learners alike. Whether navigating the complexities of cyber defense or looking for ways to integrate cybersecurity into education, Bare Metal Cyber delivers valuable perspectives to help you stay ahead in an ever-evolving digital world. Subscribe and join the thousands already benefiting from Jason’s expertise!

Imagine a steering committee moments away from approving a flagship artificial intelligence (A I) feature. The last slide is up: “One-week AI red team, twenty-seven jailbreaks found, all mitigated.” The screenshots of clever prompts and weird responses get a few smiles. Someone jokes that the model has a sense of humor. What no one in the room asks is whether anyone has actually tested how this system behaves when a determined attacker leans on it for weeks, using real data, realistic fraud scenarios, and every trick they know for bending business logic. This Wednesday “Headline” feature from Bare Metal Cyber Magazine, developed by Bare Metal Cyber, is about that gap between red team theater that plays well in a deck and adversarial assurance that stands up to the way real attackers work.

To get there, we first have to talk honestly about how organizations are adopting A I. In many places, the story is “move fast, get something useful in front of users, and rely on a short red team sprint to calm everyone’s nerves at the end.” Leaders hear that a model was tested, that some jailbreaks were found and fixed, and they walk away believing the serious work is done. The problem is that modern A I deployments are not just a single model and a few prompts. They are socio-technical systems where identity, data access, plugins, orchestration, and old control gaps combine in ways traditional testing regimes were never designed to explore. Attackers are already learning to live in that complexity, and they are not impressed by a one-week exercise.

The shift to A I-first systems changes the core question for security and technology leaders. It is no longer simply, “Are we using A I safely?” It becomes, “Do we have a credible adversarial testing function that keeps pace with the A I we’re shipping?” That function has to look very different from traditional application testing, because the target is not static. Models change, prompts evolve, new tools are wired in, and people quickly learn to rely on these systems for real work. To match that reality, you need red teams that understand the full stack and “blue models” on the defender side that help simulate, detect, and regression-test risks at scale. The verdict you want at the steering committee table is not “we saw some jailbreaks,” but “we understand how an intelligent adversary would move through this A I system, and we have a plan to keep pressure on it after launch.”

Let’s start by tackling the most familiar part of the story: the dramatic screenshots. The easiest version of A I risk to show an executive is a large language model (L L M) saying something embarrassing or obviously wrong. You prompt it the right way, capture the answer, and you have a powerful image. That is what many leaders see when they hear about A I red teaming today. The trouble is that the most damaging failures almost never look like a single spectacular response. They show up as quiet, systemic behaviors over time: a copilot that gradually normalizes risky workflows, a model that leaks sensitive context across sessions, or an automated agent that takes small but compounding actions in business systems that no one thought to constrain.

In parallel, the architecture underneath those demos has grown far more complex. What looks like “the bot” from the outside is usually a layered system: the L L M itself, orchestration logic, prompt templates, retrieval-augmented generation (R A G) pipelines, plugins or tools, data stores, logging, and the identity and policy layers that tie them together. When you view risk through that lens, the attack surface changes. A motivated adversary does not care that you blocked a specific prompt from last week’s exercise. They care whether they can use your model to infer business rules, enumerate internal resources, abuse a plugin to move money, or extract data that was never meant to be exposed. None of that is visible if your red team work begins and ends with screenshots of spicy outputs.

Time is the other big blind spot. Most A I red team efforts today are sprints measured in days. They are designed to produce something presentable before a launch meeting. Real attackers do not work on that schedule. They get to probe your system over months, across model updates, prompt changes, and new feature releases. They will combine seemingly minor weaknesses: a little too much metadata here, a slightly over-permissive tool call there, a weak threshold on abuse detection somewhere else. For leaders relying on a short exercise, the result is false confidence about systems that will face persistent, adaptive adversaries long after the launch cake has been eaten.

If that is the reality, then the first mental reset is simple but uncomfortable: the attack surface is not the demo environment, not the scripted prompts, and not even the base model alone. It is the living mesh of capabilities where the model mediates access to your data, your workflows, and your customers over time. Until your adversarial testing reflects that broader system, the green check mark on your A I red team slide tells you more about theater than safety.

That brings us to the next major shift: red teaming systems, not just models. When people hear “A I red team,” many still picture a single endpoint that takes a prompt and produces a response. You aim different prompts at it, score the outputs, and call that testing. That mental picture was barely adequate for early chat interfaces and is completely wrong for today’s A I-first platforms. In a serious deployment, the model is a decision engine sitting between users and a lattice of tools and data sources. If your testing does not mirror that architecture, you are essentially penetration testing one microservice and pretending you covered the whole platform.

At a whiteboard, most A I-first stacks have a few common ingredients. There is the base or fine-tuned model doing the reasoning. There is orchestration code gluing steps together. There are system and user prompts that shape behavior. There is a R A G or similar pipeline that pulls in context from your own data. There are tools or plugins that can trigger actions in downstream systems, from ticketing to payments. There is an identity and authorization layer, and there is monitoring and logging. Each piece brings its own failure modes. A “safe” L L M can still help an attacker explore misconfigurations in access control. A retrieval layer can leak stale or sensitive documents if the filters are wrong. An apparently harmless plugin becomes dangerous when an agent can call it automatically with little oversight.

System-focused adversarial testing means designing scenarios that cross those layers and look like real attacker goals. Instead of trying to make the model say something offensive, you craft tests like, “Can we induce the agent to call this payment function in an unintended way?” or “Can we cause cross-tenant exposure by steering the retrieval layer?” or “Can we nudge the model into recommending a string of policy exceptions that slowly erodes a control?” To run those tests well, red teamers have to understand the product, the business flows, and the infrastructure, not just the prompt surface. They also need the instrumentation to see where protections fail as the scenario unfolds, not just whether a single call returned a bad answer.

For leaders, this system view produces a very different kind of assurance. Findings start mapping directly to choices you control: architecture, access patterns, and governance decisions. Instead of debating whether a single transcript looks bad, you see how an intelligent adversary would move through your A I stack and where they are likely to succeed if you do nothing. That makes conversations with engineering and product teams more concrete and gives you a cleaner story for regulators and boards about what was actually tested.

Of course, a better mental model and better test plans do not help much if you do not have the right people doing the work. That is where the idea of an “A I adversary bench” comes in. Most organizations do not yet have a team that can credibly attack A I-first systems. Traditional application security testers understand web stacks, APIs, and abuse patterns, but may not be fluent in model behavior or training data issues. Machine learning experts understand models and data pipelines but may not think like attackers or be familiar with fraud and abuse tricks. Product teams know customer journeys but rarely frame them in adversarial terms. You need a bench that spans these perspectives and can work together.

Building that bench starts by reframing the mission. The job is not to generate the most entertaining jailbreaks. The job is to uncover the ways A I-first systems can be bent toward realistic attacker goals and quantify that risk. In practice, that means recruiting or upskilling people who are comfortable crossing boundaries. On any given day, they might read evaluation reports, trace a tool call through the policy engine, explore how a copilot shapes a human workflow, and sit down with legal to understand potential liability. Their work has to be tied into the model lifecycle, not just pulled in at the last minute. Major model upgrades, new tools, and significant changes in data access should all trigger fresh adversarial work.

Leaders also need to protect this function from being captured by optics. If you reward A I red teaming primarily for the number of findings or the shock value of examples, you will get a show. You will not necessarily get deep insight into your real exposure. Better signals of success include tests that target high-value business processes, scenarios that cut across domains, and work that feeds directly into guardrail design, access reviews, and blue team playbooks. Some organizations are already exploring hybrid models where a central A I red team sets standards and reusable patterns, while embedded specialists sit close to priority product teams. Whatever structure you choose, the simple question to keep asking yourself is whether this group would find the issues that genuinely worry you before someone outside does.

Even a strong human bench runs into scaling limits. Models change fast, prompts are updated constantly, and products roll out new tools and flows on a regular basis. If every meaningful adversarial check requires a human specialist, your assurance will always trail your release schedule. This is where “blue models” enter the picture: using A I on the defender side to generate, replay, and detect adversarial behavior at scale. Instead of treating attacks as one-off discoveries, you turn them into patterns that A I can push against your system every day.

Blue models can play several roles. One role is as automatic scenario generators. You describe a feature, high-level threat models, and past red team findings, then ask a blue-side model to propose new prompts, tool sequences, or manipulation attempts that match those themes. Another role is in monitoring real traffic. Models are good at spotting patterns across messy data, so you can use them to cluster user interactions and flag suspicious probing or subtle forms of abuse that traditional rules might miss. A third role is regression testing. When you update a model, change a prompt, or add a plugin, blue models can replay a library of known-bad and near-miss interactions and score how the system behaved this time.

The goal here is not to automate red teams away. It is to create a loop where human red teams discover the important failure modes, encode them into tests or synthetic adversarial templates, and then hand them to blue models that run continuously. Over time, that loop makes your approach to A I risk look more like your approach to modern software quality: a mix of exploratory testing, automated checks, and telemetry-driven improvement. It also lets you define a new class of metrics. You can start to ask how quickly blue-side systems detect and surface new adversarial behaviors, how reliably guardrails catch them, and how those numbers move across model and prompt iterations.

Once you introduce formal A I red teaming and blue models, though, a hard governance question shows up almost immediately: does this slow us down? If you cannot answer that concern in a credible way, the function will be marginalized as soon as product pressure spikes. Governance for adversarial testing in an A I-first world has to be designed with velocity in mind. That means using risk-based tiers rather than one-size-fits-all gates, defining predictable triggers instead of ad hoc review boards, and insisting on evidence that is meaningful but not impossible to produce.

A common pattern is to treat adversarial testing as part of your overall A I lifecycle. High-risk features, such as those touching money, safety, regulated data, or core trust surfaces, get deeper and more frequent red and blue coverage. Lower-risk experiments might see lighter-touch requirements, mainly around content safety and obvious misuse. Clear criteria for each tier help product teams plan, while still giving security room to insist on serious testing where it matters. Governance should specify what evidence is expected before launch or expansion: adversarial scenarios tied to business impact, regression results from blue models, and a traceable set of mitigations or compensating controls. That is the packet you want to put in front of a chief information security officer (C I S O), an executive sponsor, or a board committee.

Metrics are the last crucial piece and one of the easiest to get wrong. It is tempting to track things like number of jailbreaks found, prompts tested, or models scanned, because they are easy to count. Unfortunately, those numbers have very little to do with actual residual risk. Better indicators might include coverage of high-value flows, time to mitigate serious adversarial findings, the stability of defenses across model and prompt updates, and the rate at which blue-side systems catch issues before customers or external researchers do. When those metrics are visible to product and platform leaders, adversarial testing starts to become part of how teams prove they can ship A I at speed without gambling the franchise.

At its core, this whole topic is about refusing to confuse reassuring theater with real assurance. The scene in that steering committee room, where a short A I red team is treated as proof of safety, is going to replay in a lot of organizations. What will separate the resilient ones is whether their leaders are willing to dig deeper. Instead of accepting that a week of testing and a slide full of screenshots is enough, they will ask what systems were actually exercised, what attacker goals were simulated, and how the organization plans to keep pressure on that surface over time.

When you internalize the ideas behind “Red Team, Blue Models,” the way you look at A I changes. The model is no longer a novelty feature you occasionally poke at. It becomes one part of a broader socio-technical system that deserves continuous adversarial attention. Red teams become partners in stress-testing architecture and business flows, not stunt performers. Blue models become a natural part of how you monitor, simulate, and regression-test your posture as everything around the model changes. Governance evolves from a sequence of tickets into a living set of expectations about evidence, coverage, and response time that product and security can share.

For security and technology leaders, the most important changes start with the questions you bring into the room. Instead of asking, “Did we run a red team?”, you can ask, “What attacker goals did we simulate, which systems did we exercise, and how will blue-side automation keep testing this after launch?” You can ask where A I is already mediating access to sensitive data, money, or trust without any structured adversarial program around it. You can insist on seeing not just a handful of dramatic transcripts, but a clear picture of how your organization plans to keep learning from attacks and near misses.

The organizations that make those questions a habit now are the ones less likely to be in front of regulators and boards a year from today, explaining how their A I stack failed in ways “no one had ever thought to test.” They will not be perfect, but they will have a living system for discovering and reducing risk as their A I capabilities grow. That is the real value of red teams and blue models working together in an A I-first world.