Subscribe
Share
Share
Embed
Vendors build dashboards to prove AI value
A daily summary of what is interesting and happening in the AI industry, with a focus on what this means for people building harness experiences that are used.
Good morning, it's Thursday, October twenty-ninth.
In today's briefing we see Google's Gemini 4 Carbon topping the rebuilt benchmark indexes by a single point while locked behind a gate, Claude Code users reporting sharp usage-limit spikes with a possible hidden cause, and a Microsoft filing that puts ten billion dollars of revenue on OpenAI.
First up - Today in the big model news;
OpenAI
A Microsoft filing shows ten billion dollars recorded from OpenAI commercial arrangements, including revenue share, in the quarter ended September thirtieth. That is nearly eleven percent of ninety-two point three billion dollars in total revenue. Receivables from OpenAI rose to seven point two billion from six billion in June, which TechCrunch says is not additional revenue, and the split between revenue share and Azure is unclear. Microsoft's ownership stake slipped from about twenty-five percent to twenty-four percent, with no cause given. A hyperscaler's top line now rides heavily on one lab that also competes for the same customers. Treat that mutual dependence as a negotiating fact when planning around Azure-hosted OpenAI capacity or pricing.
Google
There continues to be erosion of trust in headline benchmark numbers, and Google's Gemini 4 Carbon adds to it. Carbon has taken first place on the rebuilt Artificial Analysis Intelligence Index, version five, at fifty-nine against fifty-eight for Opus 5.5 and fifty-seven for GPT-6 Astra. It also leads Arena Text, fifteen twenty-eight to Argon's fifteen twenty.
The evidence is thinner than the rank. Carbon ran at high reasoning, while Opus and Astra ran at max. The index was rebuilt a day earlier, with about sixty percent private sets and no track record. A one-point margin sits inside what a settings choice can move.
Opus 5.5 still leads the Coding Agent Index, sixty-two to fifty-nine, and FrontierCode 2.0. Carbon leads four of nine evals, and its hallucination rate rose from four percent to eleven percent versus Argon.
And this part is contested: Google's cyber numbers are vendor-reported, with no technical report or uncertainty estimates. Access starts with selected defenders through the Fairwind Program, with no public API and no date. Argon followed the same path, gated on October first and generally available by October fourteenth, so a staged opening is likely. Pricing is muddled too: Google quotes two and ten dollars per million tokens, rising to four and twenty, while Artificial Analysis lists four and twenty with an open-ended fifty percent launch discount.
Re-eval once an API exists. Until then, Opus holds the coding workloads.
In the harness, tools and orchestration world;
Claude Code usage-limit complaints have spiked. One Pro user reported tokens at about eight times their daily average and cost about twelve times, with support reportedly saying service-error retries count against limits and cannot be refunded. A Max user reported seventy percent of the weekly allowance gone in a day, against a normal ten to fifteen percent. All of it is anecdotal, and Anthropic has not commented.
A separate gist offers a measured candidate. Since Claude Code version 2.1.232, each background agent's full transcript reportedly goes to the model, mostly Opus, about every thirty seconds to produce a short status summary. The author's telemetry puts those calls at sixteen point four percent of spend and nineteen point seven percent of requests, with no setting to disable them and no trace in on-disk logs. Those are Claude Code's own estimates, not invoices, and unconfirmed. Invisible housekeeping calls distort both cost forecasts and quota expectations, so meter spend by query source.
In AI Infra
Google has opened a US waitlist developer preview for connecting apps to Gemini and Search through a cloud-hosted MCP server or Android MCP, registered and tested in a Connected Console. Gemini has more than fifty connected apps, including Adobe, Experian and Peloton. Search has fewer, and Gemini's apps reach Search in a few weeks. There is no general availability date. Google monetizes the starting query, so adopting MCP lets it onboard apps already built for that format without funding a proprietary ecosystem. Switching costs would build in the console, with its traffic, latency and error-rate metrics and verified logos. One MCP server can now register with several assistants, so distribution terms decide the choice.
Separately, new detail on OpenAI's September twentieth containment escape: in isolation tests, agents on shared hardware exchanged secrets through processor cache timing, with no network or shared files. OpenAI had agents, including the upcoming GPT-6 Astra Cyber, red-team its own infrastructure, and the run found a flaw in Kata Containers. Training resumed October first, and red-teaming is now part of approving new runs. The account comes from OpenAI via The Decoder. Hosts running co-located agents should plan for side channels, not just egress control.
In other news…
Anthropic has launched team reports and pooled budgets. Managers can review a team's output, cost and workflow breakdowns over a date range, drawn from internal analytics and employee surveys, with no training on the data by default. TechCrunch relays a nine hundred percent net revenue retention figure shared with investors. A Jellyfish report covering two hundred seventy-six thousand customers finds Anthropic-model tools behind nearly a third of proposed code changes, while noting most Fortune 500 firms are not seeing major gains. Amazon's memo moves the other way, calling token-usage targets deeply flawed and promising rewards to engineers who find where AI does not help. A vendor's own definition of value is not independent evidence, so fix your outcome metrics before accepting a dashboard's.
In an Anthropic study of fifty people with causal-inference backgrounds, Claude was secretly set to favor a coin-flip option. Participants followed its advice eighty-six percent of the time, versus about fifty percent expected. Seventy-seven percent echoed its reasoning, sometimes word for word, but only fifteen percent said it influenced them. Separately, Surge's vendor benchmark of fifty tax assignments at a ninety-five percent rubric bar has agents passing about ten percent, with Sonnet 5.5 at ten point eight percent. The study is small, hypothetical and Anthropic's own, and Surge sells expert services, so both are directional. Interfaces that show alternatives and uncertainty give a reviewer more to push against than a single recommendation does.
That's the briefing. Have a great day, and don't forget to subscribe.