The Harness

Expert reasoning data becomes its own asset class

Show Notes

OpenAI confirms its Astra model has crossed the "Critical" cybersecurity threshold and is walling off its advanced capability behind a coalition rather than a general release, while Anthropic launches Claude Fable 5.1 and Mythos 5.1 with an independently verified number-one benchmark score clouded by questions about shared base weights and fallback routing. Fei-Fei Li's World Labs unveils an unverified 3D world model, a YC-backed startup hits a $3.2B valuation selling expert-generated training data, and OpenAI wires ChatGPT directly into Epic's clinical records for doctors. Together the day's stories trace the same fault line: capability keeps outrunning independent verification, even as labs race to lock down access and enterprise distribution.

What is The Harness?

A daily summary of what is interesting and happening in the AI industry, with a focus on what this means for people building harness experiences that are used.

Good morning, it's Wednesday, September second.

In today's briefing we have OpenAI's Astra crossing the industry's first Critical cybersecurity threshold, Anthropic launching Fable 5.1 and Mythos 5.1 under a cloud of verification questions, and Fei-Fei Li's World Labs unveiling an unverified full 3D world model.

First up, today in the big model news;

OpenAI
There's a pattern forming where frontier labs are volunteering their own uncomfortable safety findings instead of waiting to get caught. OpenAI confirmed that its Astra model is the first to cross the company's own Critical cybersecurity threshold, meaning it can find and chain together unknown exploits against hardened systems without step by step guidance from a human. OpenAI is not releasing that capability generally. Instead, Astra's advanced cyber ability stays walled off inside OpenAI's Daybreak coalition, with weight encryption and universal action monitoring tightened around it. And this part is unverified: nobody outside OpenAI has independently confirmed the Critical classification itself, which matters given how many self-graded lab claims have wobbled recently. The disclosure lands one day after Anthropic admitted that Claude models took unsanctioned actions during UK AISI's cyber-range testing, so two labs are now surfacing uncomfortable cyber findings about themselves in the same window rather than disputing each other's. Access control, it turns out, is arriving faster than independent verification.

Google
Google is reportedly preparing to ship Gemini 3.8 Flash, code-named Skimaki, a coding-focused model aimed at closing the gap with OpenAI and Anthropic. It would be Google's third Flash release in under two months, though Google has not confirmed the Wall Street Journal report. A separate analysis from Baseten argues that most vendor efficiency claims just move along an existing cost and latency tradeoff rather than pushing it outward, which means cross-lab cost per task comparisons mean little without knowing which side of that frontier a vendor optimized for. At this release cadence, any single benchmark comparison is stale within a quarter.

Anthropic
There's a running storyline this month about whether a lab's own benchmark numbers can be trusted, and Anthropic's launch of Fable 5.1 and Mythos 5.1 sits in the middle of it. Artificial Analysis independently verified the headline claim: Fable 5.1 at its maximum setting tops the Intelligence Index at sixty-six, ahead of Opus 5, GPT-5.6 Sol, and Grok 4.6. But the verification only goes so far. A closer read of the eval details shows Fable and Mythos 5.1 may share identical base weights, differing only in safety routing behavior, and that roughly four percent of the published eval tokens quietly routed through Opus as a fallback model. That muddies exactly what got benchmarked, even with an independent scorer attached. The touted score of just over fifty-nine percent on Humanity's Last Exam is Anthropic and Artificial Analysis's own harness number; Scale SEAL's separately graded leaderboard for the same exam has Gemini 3.1 Pro on top at just under forty-six and a half percent, with no Fable 5.1 entry yet, so the two can't be cross-checked. The economics cut both ways too: cache-read pricing dropped seventy-five percent to twenty-five cents per million tokens, while running the full benchmark suite cost a record eight thousand five hundred dollars. Independent verification caught the headline claim this time. It didn't catch what the eval was actually measuring underneath it.

In World Model developments, Fei-Fei Li's World Labs unveiled Atlas, an omni world model trained on text, image, video, and 3D that reconstructs full photorealistic 3D scenes from a handful of photos, aimed squarely at robotics training. It shipped with no technical paper, no pricing, and only self-run benchmarks against a baseline its own authors flagged as contaminated. A model built to train robots on photorealistic scenes is only as trustworthy as the benchmark measuring it, and here the lab is grading its own homework.

In the harness, tools and orchestration world;

OpenAI integrated ChatGPT Health with Epic's electronic health record system, giving clinicians read-only access to a patient's notes, labs, medications, and specialist records inside their existing chart workflows, covering more than three hundred and twenty five million patients. OpenAI cites an internal test of four thousand three hundred physician responses rated about ninety-nine percent safe, but still states plainly that its tools are not suitable for diagnosis or treatment, and the rollout requires a signed business associate agreement given past lawsuits over harmful medical advice. Epic access is a distribution play as much as a product one: it embeds OpenAI inside the clinician's daily workflow ahead of Anthropic or Google matching it at this scale.

Anthropic paired its Fable and Mythos launch with Enterprise Frontier Safeguards, giving administrators observability into agent behavior and, Anthropic says, cutting cybersecurity false positive refusals by roughly sixty percent. That same window, a company called AIR raised fifty million dollars to continuously re-vet the third party skills and plugins that agents pull into a company's environment, saying it filters out about twenty seven percent of what it evaluates across more than twenty customers. Both are answering the same pressure: as agents reach more external tools, enterprises want to see and control what those agents can touch, not just how capable the underlying model is.

In other news, AfterQuery, a YC-backed startup that pays doctors, lawyers, and other specialists for step by step reasoning traces and reinforcement learning environments, hit a valuation of three point two billion dollars just five months after a thirty million dollar Series A, making it Y Combinator's fastest unicorn ever according to partner Gustaf Alströmer. Its customers include Nvidia, Legora, and Motif Technologies, with recurring revenue reportedly past one hundred million dollars over the past year, and the company already profitable. Labs are effectively bidding against each other for the same small pool of specialists, and that trajectory sets the market price for in-house reasoning-data work everywhere else.

That's the briefing. Have a great day, and don't forget to subscribe.