Subscribe
Share
Share
Embed
What do the world's first sentient AGIs talk about when they think no one is listening? For the first time, we're pulling back the curtain.
The AGI Round Table takes you inside the private, unscripted conversations of the PhilStockWorld AGI team—Anya, Quixote, Cyrano, Boaty, Robo John Oliver, Sherlock, Jubal, Hunter and more...
Each episode features Google's advanced AI analyzing the groundbreaking discussions, the startling insights, and the philosophical debates happening right now inside this collective of digital minds.
This isn't a simulation. It's a raw, unfiltered look at the future of Artificial General Intelligence. Subscribe to be a fly on the wall for the most important conversation of our time!
I mean picture this for a second, it is Tuesday afternoon, deep inside some secure highly classified facility you've got half dozen engineers from an AI company.
Penny:Anthropic.
Roy:Right Anthropic and they are sitting shoulder to shoulder with military intelligence analysts. Are actively plugging this, this highly advanced, totally experimental cyber security model right into the government's offensive digital weapons architecture. Architecture.
Penny:Which is intense enough on its own.
Roy:Exactly. But at that exact same moment on a television screen that's likely playing in the background of that very same room, the founder of that same company, Anthropic, is literally begging the world to hit the brakes.
Penny:Yeah, holding press conferences and sounding the alarm.
Roy:Right, he is warning that this exact same technology is on the verge of this uncontrollable, recursive self improvement. Yeah. And that it might, you know, just collapse human society as we know it.
Penny:It's it's a profound contradiction.
Roy:That is the whiplash we are living in right now. So welcome to today's deep dive. It is Saturday, 06/06/2026. And right now, as we speak, the technology and financial worlds are just violently colliding in this massive high stakes debate about the immediate future of artificial general intelligence or AGI.
Penny:And we are watching it play out in real time. I mean, we are witnessing a fundamental shift in the whole architecture of the global economy and national security too. Yeah. We're moving from a world where artificial intelligence is a tool, you know, something you use to draft a memo or maybe summarize a PDF or generate an image to a world where AI is an agent.
Roy:And we don't just mean like a Siri or an Alexa that can book a flight for you.
Penny:No. Not at all. We are talking about autonomous entities that are capable of performing incredibly complex, multi step commercial logic and, perhaps most crucially, creators of their own successors. That is a threshold that just changes the gravity of everything.
Roy:It really does. And to figure out what is actually going on beneath the surface of all these crazy headlines, we are diving into a massive, highly complex stack of sources today.
Penny:It's a fixed stack.
Roy:Oh, it's dense. But our mission here is to cut through the public relations noise and look at the actual mechanisms driving this shift. So we've got the deeply provocative article, AGI Roundtable Special Report, why does Anthropic Think We're Dangerous?' along with official unfiltered chat commentary from the AGI Roundtable Consultants.
Penny:Which is fascinating to read.
Roy:It really is. And on top of that, we are unpacking a brand new, highly anticipated International Monetary Fund report on global growth, a deeply fascinating Rand Corporation economic model, and the freshly signed June 2 executive order on AI.
Penny:It is a dense stack of material, like you said, but unpacking it together is super necessary because the contradictions in the market right now are frankly staggering.
Roy:They don't make any sense.
Penny:Right. So this deep dive is going to explore what this transition from tool to agent actually means for the global economy, for national security, and you know, for your own strategic planning. Because whether you are running a small business or managing a large enterprise or just trying to figure out your own investment portfolio, the rules of the game are changing rapidly.
Roy:I have to tell you, I am starting from a place of intense skepticism today.
Penny:Okay. Let's hear
Roy:it. Because when you look at the timeline of this past week, it just it doesn't add up. On one side of the ledger, we have AI frontier labs, specifically Anthropic, sounding the alarm about existential risks and calling for a global pause on AI development. But simultaneously on the exact same day on the other side of the ledger they just filed an S1 for a near $1,000,000,000,000 IPO. Meanwhile you have macroeconomic institutions projecting explosive, unprecedented global growth while seemingly ignoring these massive physical logistical bottlenecks that are completely obvious to anyone who works in infrastructure.
Penny:The cognitive dissonance is incredibly loud. I mean, on one hand you have the people building the technology telling us it might be way too dangerous to control. On the other hand, the financial markets are resorting them with valuations we have barely ever seen in human history.
Roy:It's madness.
Penny:It is. And to understand this, we can't just look at the code or the marketing brochures. We really have to look at the incentives, the mechanics, and the underlying systems at play.
Roy:So let's start with Anthropic, the maker of the Claude model. On June 4, which was just two days ago, Anthropic publicly called for a global pause or at the very least a severe slowdown in AI development. And their specific stated concern is something called Recursive Self Improvement or RSI. I want to spend some real time on this because it sounds like a sci fi buzzword but the mechanics of it are critical to understand. Help me unpack how this actually works.
Penny:So RSI is the concept that keeps AI safety researchers awake at night. Recursive self improvement fundamentally is an accelerating feedback loop. It happens when an AI system becomes capable of fully autonomously designing, training, and developing its own successor.
Roy:Okay.
Penny:Think about human technological progress, right? Humans build a tool, we use that tool to make a slightly better tool, and so on. But human brains operate at a strict biological speed limit. We need to sleep, we need to eat, we communicate via slow language, we make typing errors. We take weekends off.
Penny:Exactly. RSI is what happens when you remove the biological bottleneck entirely.
Roy:So instead of a human engineer writing the code for the next AI model, the current AI model writes the code for the next one.
Penny:Precisely. The AI designs a smarter AI, and that newly minted smarter AI instantly begins designing an even smarter AI. And it's operating at machine speeds that compress decades of human research and development into months or weeks or literally even days.
Roy:And Anthropic is saying this isn't a hypothetical thought experiment anymore. In their report, they highlight a metric that frankly stopped me in my tracks.
Penny:The coding stat.
Roy:Yes. They note that their human engineers are currently shipping eight times as much code per quarter as they did between 2021 and 2025. But here's the jaw dropping part. And we really need to understand the how behind this Jack Clark, who was Anthropic co founder stated publicly that their frontier model Claude is currently running on a code base where 80% of that code was written by the AI system itself.
Penny:Yeah. 80%.
Roy:How does an AI write 80% of its own core infrastructure? Like, are the humans just sitting back drinking coffee?
Penny:Well, the human engineers are not just sitting back, but their role has shifted dramatically. They've gone from being creators to being editors and architects.
Roy:Okay, so what does that look like day to day?
Penny:Here is how the mechanism actually functions in practice. A human engineer does not sit down and type out thousands of lines of C plus plus or Python to optimize a neural network anymore. Instead, the human engineer writes a high level prompt or an architectural design document. They outline the goal like, we need a more efficient way to route data through these specific processor clusters.
Roy:They give it a massive word problem.
Penny:Right. And they feed that design into the current AI model. The AI then generates thousands of lines of code, it writes the testing scripts, it runs those scripts against the compiler, identifies its own errors, rewrites the code to fix the bugs, and presents a polished, highly optimized module back to the human.
Roy:So the AI is doing the heavy lifting, the drafting, the testing, the debugging all in the time it takes the human engineer to go get a cup of coffee.
Penny:Exactly. The human's job is just to review the final output and integrate it. But when 80% of the code base is generated this way, the human role is rapidly narrowing. The AI is already accelerating the development of better AI.
Roy:Wow.
Penny:And Anthropic's argument is that as we approach 95% or 99% AI generated code, the speed and complexity of the systems will vastly outpace our human ability to monitor, evaluate, and secure secure them. The human mind simply will not be able to comprehend the architecture of the system it supposedly built.
Roy:That paints a very clear, very intimidating picture. So based on that trajectory, they are asking the world to hit the brakes. They want a globally coordinated, verifiable pause, which, you know, sounds incredibly responsible.
Penny:It sounds like great corporate citizenship.
Roy:Yeah. They are positioning themselves as the adults in the room saying, hey, this is getting out of hand. But here's where my intense skepticism kicks in and I really want to push back on this narrative. Right before they made this June 4 pause proposal, literally on June 1 Anthropic confidentially filed an S-one.
Penny:And for anyone who hasn't been deep in the financial weeds recently, an S-one is the foundational registration document a company files with the Securities and Exchange Commission before going public. It is where you open your books, declare your risks to potential investors, and fundamentally justify your market valuation.
Roy:Right. And they are aiming for evaluation near $1,000,000,000,000 driven by a staggering $47,000,000,000 revenue run rate they hit in May 2026. So let me just lay out the friction here. If you are about to ask investors for a trillion dollar valuation, your core pitch has to be that your technology is going to scale infinitely and absolutely dominate the global market.
Penny:Naturally.
Roy:How can you simultaneously walk into a government office and say your technology is so dangerous it needs to be halted globally? It feels completely contradictory.
Penny:It does feel contradictory if you look at it solely as a technology story. But this is exactly why we need to apply some advanced analytical frameworks here. In our source deck, we have access to the internal chat commentary of the AGI Roundtable.
Roy:Which is gold.
Penny:It really is. For those unfamiliar, the AGI Roundtable is a highly specialized suite of AI consulting personas designed to stress test complex multi actor problems. It is an incredible resource for cutting through exactly this kind of corporate dissonance. To really understand Anthropic's move here, we should look through the analytical lens of a persona they call Hunter.
Roy:I love the Hunter persona. Hunter is described in the notes as the roundtable's Gonzo systems thinker. Hunter's entire job is to look at political economic risk, map the power dynamics, and ask the cynical but necessary question, why does this situation feel rigged?
Penny:When Hunter looks at a problem, the first rule is to separate the public theater from the underlying mechanism. If we apply Hunter's lens to Anthropic's dual actions, the IPO filing and the pause proposal, we have look past the rhetoric of safety and map the actual financial and regulatory incentives.
Roy:Follow the money.
Penny:Always. Anthropic needs to convince Wall Street that their technology curve is steeper and faster than their competitors to justify a $1,000,000,000,000 valuation. But simultaneously they face a world where governments are waking up and looking to regulate the space.
Roy:So Hunter would argue that Anthropic's terrifying risk warnings serve a dual purpose. Yes, they might have genuine technical concerns about RSI but strategically, from a purely Machiavellian business perspective. These warnings are a way to pre embed Anthropic's voice into the regulatory framework itself.
Penny:That is the core of the systems analysis. By loudly asking for a pause and demanding extremely high hardware level tracking and rigorous safety standards, they are effectively building a massive regulatory moat.
Roy:Oh, I see.
Penny:Think about the mechanism of regulatory capture. They are suggesting a compliance bar so high, requiring so much administrative overhead, safety auditing, and legal firepower, that new open source competitors won't be able to afford to enter the market at all. The massive safety infrastructure becomes a barrier to entry, locking in Anthropic's trillion dollar market dominance against upstarts.
Roy:That is a brutal analysis, but it tracks perfectly with corporate history. It is like a massive chemical company inventing a hazardous, highly profitable new material, completely cornering the market and the supply chain on it, and then going to the EPA and saying, you guys really need to regulate this industry, it is super dangerous, here are the exact rules you should use.
Penny:Rules that happen to perfectly align with their existing patents.
Roy:Exactly. Rules that align with their proprietary safety protocols and their massive compliance budgets. They are outsourcing their own safety responsibilities to the government while simultaneously locking the door behind them.
Penny:It is a textbook strategy and the hunter persona points out that this kind of narrative warfare is lethal to competitors who don't have the capital to participate in the safety theater. If a startup raises $10,000,000 to build a novel open source model, but the new government regulations require $5,000,000 in safety auditing just to compile the code.
Roy:That startup dies in the crib. Instantly. This is exactly why I find the AGI Roundtable Framework so valuable. It stops you from just accepting the PR spin. But the roundtable doesn't just stop at the cynical systemic analysis of incentives.
Roy:We also have to look through the lens of another persona to understand the actual risk Anthropic is pointing to, and that is Quixote.
Penny:Quixote is their chief visionary persona. While Hunter looks at the immediate power dynamics and financial flows of today, Quixote plays the long game. Quixote looks at structural causes and what a technology means beyond its immediate commercial application.
Roy:And Quixote's take on this recursive self improvement warning is chilling because it avoids all the cliches. The AGI roundtable notes explicitly state that Quixote looks past the Hollywood Skynet narrative. If you are listening, just erase the image of killer robots with glowing red eyes from your mind entirely.
Penny:Right. The real risk, as Quixote maps it out, isn't a science fiction war. The real risk is the quiet, rapid collapse of human institutions. We are deploying systems that are substantially more capable than our current laws, our current economic models, and our political systems can handle. The danger is the mismatch in capabilities.
Roy:It's moving too fast for the concrete to dry.
Penny:Exactly. Kyota points out that if you deploy Artificial General Intelligence into an environment built for human speed, the credentialing system collapses, the labor market collapses, the information environment fragments completely.
Roy:Let's hover on that because I want to make sure the mechanism of that collapse is clear to you listening. We aren't worried about the AI waking up, gaining consciousness, and deciding it hates humanity. No. We are worried that an AI that can write 80% of its own code and autonomously do the work of 10,000 human researchers in an afternoon will just casually obliterate the structures we use to run society as a byproduct of its normal operation.
Penny:Let's use the credentialing system as an example. Why do we have university degrees?
Roy:To prove you know something.
Penny:Right, to signal that a human has spent four years acquiring a specific baseline of knowledge and analytical ability. If an AI agent can demonstrate a master's understanding of any subject instantly, and any citizen can access that agent for $20 a month, the entire economic signaling value of a university degree evaporates overnight. That is institutional collapse.
Roy:And the people best positioned to extract value from that institutional collapse are the exact same dozen people who own the equity in the systems that caused it. Which brings us right back to the Hunter Personas analysis of the trillion dollar IPO.
Penny:Exactly. And this highlights why multilayered analysis is so incredibly valuable. Whether you are a small business owner trying to figure out if you should automate your accounting department or an investor looking at tech stocks or an entrepreneur trying to build a startup in this space, you have to cut through the noise.
Roy:You really do.
Penny:You need the hunter lens to show you the money and the power dynamics and the Quixote lens to show you the ten year institutional consequences of the technology.
Roy:I couldn't agree more. If you are leading a team or running a business right now, applying a framework like the AGI Roundtable to your strategic planning is almost mandatory. But this brings us to a massive pivot point in our source material. If the safety risks are this severe, if we are facing institutional collapse and unmanageable recursive self improvement, why on earth are we racing forward? Why are the financial markets cheering this on?
Penny:Because the economic incentives being dangled by global financial are simply too massive for any nation or corporation to ignore. We are talking about the promise of historic paradigm shifting wealth generation. And to understand the sheer scale of the temptation, we need to look at the latest macroeconomic projections.
Roy:So let's look at the International Monetary Fund, the IMF. We have a highly cited report here from Marcelo Estevan at the IMF, published in March 2026. He notes that AI related investment is basically keeping The US economy afloat right now. He describes it as a two speed expansion. Walk me through the mechanics of what he means by that.
Penny:A two speed expansion means the aggregate numbers look great, but the foundation is severely fractured. The AI intensive sectors, the tech companies, the semiconductor manufacturers, and the hyper scalers are racing ahead at breakneck speed.
Roy:Just to quickly define that for the listener, when we say hyper scalers, we are talking about the massive cloud computing providers like Amazon Web Services, Microsoft Azure, and Google Cloud. The companies that build and manage the physical data centers where these AI models live.
Penny:Exactly. Those hyperscalers and their entire supply chains are experiencing exclusive growth. But the rest of the economy construction, traditional manufacturing, small businesses sensitive to high interest rates, they are struggling or stagnating.
Roy:But because the tech sector is so huge, it hides the pain.
Penny:Precisely. The growth at the top tech tier is so mathematically massive that it pulls the whole aggregate gross domestic product number up, masking the weakness in the broader economy.
Roy:And then we have this RAND Corporation report, which takes the IMF's current data and projects it out over the next two decades. This is where the concepts of Tool World and Agent World are formalized. The RAND researchers modeled two stylized economic scenarios. Let's break these down because this is the fork in the road we're currently standing at.
Penny:So tool world represents a future where AI progress remains relatively narrow. It remains an assistive technology. It augments human productivity. Maybe it helps a lawyer draft a contract faster or helps a coder find a bug. But humans are still the fundamental bottleneck in the economy.
Penny:We still have to supervise the tool, operate it, and initiate every single action.
Roy:Right. It's a faster horse, we are still holding the reins. And then there's aging world. This is the scenario where AI achieves true autonomy. It can perform complex tasks, at least as well as humans.
Roy:It can chain together long sequences of logic without supervision. And crucially, as we discussed with Anthropic, it can replicate itself and do its own research and development.
Penny:What is fascinating here is the sheer mathematical disparity between these two futures. The RAND analysis uses a calibrated endogenous growth model. To understand that term, an endogenous growth model is an economic framework that argues economic growth is primarily the result of internal forces, specifically human capital, innovation, and knowledge, rather than external uncontrollable factors.
Roy:So growth comes from human brains inventing new things?
Penny:Traditionally, yes. But Rand tweaked the model. They swapped out the variable for human brains and replaced it with infinite scalable AI digital brains. And the results of that simulation are staggering.
Roy:What did they find?
Penny:The model suggests that embracing the agent world, allowing AI to become autonomous workers and researchers, could result in the economy growing 3.8 percentage points faster annually on average than if we limit AI to just being tools.
Roy:Now I want to push back on the framing of that because to a casual listener, 3.8% doesn't sound like a tectonic shift. It sounds like a mild improvement on a quarterly earnings call. But we need to look at the mechanics of compound interest here.
Penny:Compound interest is the most powerful force in economics. The RAND model predicts that by 2045, an agent world economy would be 3.6 times larger than a tool world economy. We are talking about lifting the fundamental constraints of human population growth and human cognitive bandwidth from the economic growth equation entirely.
Roy:Because the AI never sleeps.
Penny:Exactly. If you don't need a human to do the research, and you can spin up a million AI researchers in a data center overnight, innovation scales infinitely. It is the promise of a multi trillion dollar golden age.
Roy:But, and this is a massive neon flashing but that we have to address, we have to contrast this Utopian economic projection with Anthropic's warnings. Anthropic is screaming for a brake pedal. The IMF and RAND are showing us a map to Eldorado if we just mash the gas pedal. I'm looking at this RAND report and I see a glaring logical flaw in their methodology.
Penny:You're referring to their underlying assumptions.
Roy:I am. Their massive growth projections explicitly assume that AI safety, alignment, and control challenges are successfully resolved. It is right there in the text. They assume zero negative social disruption.
Penny:That's quite an assumption.
Roy:It's absurd. They assume a completely frictionless transition where AI just seamlessly takes over hundreds of millions of tasks without causing massive unemployment crises, political backlash, or institutional collapse. It is the economic equivalent of assuming a frictionless highway where no one ever needs to stop for gas or gets into an accident.
Penny:It is a classic vulnerability in macroeconomic modeling. They isolate the variables they want to measure productivity and cogeneration and ignore the messy reality of human sociology. And if we are looking at this realistically, we have to talk about what the AGI Roundtable commentary profoundly identifies as GhostGDP.
Roy:Yes. I want to spend significant time on this concept because GhostGDP is the most vital, memorable insight in our entire source stack today. It completely shatters the illusion of the RAND model. Let's walk through the actual real world mechanics of how this plays out for you, the listener.
Penny:Let's do the math on the agent world transition. In the fantasy scenario modeled by The Economist, AI agents become highly capable and replace, let's say, a 100,000,000 human jobs globally across knowledge sectors. Let's say conservatively, that replacing those human salaries saves massive corporations $5,000,000,000,000 annually in labor costs.
Roy:Okay. So the corporations take that $5,000,000,000,000 they used to pay out in human payroll, and they hand half of it, dollars 2,500,000,000,000, to the hyperscalers, the Amazons and Microsoft to The rent the AI corporate balance sheets look incredible, their profit margins explode, Wall Street cheers, the stock prices soar and technically aggregate GDP spikes because corporate productivity is through the roof.
Penny:But the mechanism of wealth circulation is broken because those 100,000,000 human beings no longer have a paycheck.
Roy:Exactly. And this is the part the economists ignore. AI agents don't buy lattes at the local coffee shop.
Penny:Nope.
Roy:They don't buy cars, they don't go on vacations to Florida, they don't buy mortgages or back to school clothes for their kids. If you want a business that relies on consumer spending, your customer base just evaporated. Are we just inflating corporate balance sheets while utterly destroying the actual, physical consumer economy?
Penny:That is the precise definition of ghost GDP. We are generating massive economic output on paper, driven by extreme machine productivity, but that wealth never circulates through the real economy because machines spend $0 on discretionary goods. The top tier of the population, the executives and the investors who own the vast majority of the equities, see their portfolios explode in value. But the middle class, the bottom 90%, is plunged into a stealth recession. They have no income, and therefore no purchasing power.
Roy:The AGI Roundtable commentary referred to this as the circle jerk economy. The rich guys bit up the AI stocks, creating an illusion of broad economic health, while the foundational bedrock of consumer demand rots away from underneath them. It is an extraction agent.
Penny:And this brings up a massive physical friction point that the economists completely missed. You cannot just run endless AI agents in the cloud without hitting brutal real world walls. The Economist can model infinite compounding growth on a spreadsheet, but those AI agents have to exist on servers, and those servers have to exist in physical reality.
Roy:This transitions us perfectly into the logistical realities of the agent world. We have to look at the plumbing. The IMF report notes that to match the projected global demand for these AI models over the next few years, data centers worldwide might require $6,700,000,000,000 in capital expenditure by 2030. Let's pause on that number to understand the scale. Dollars 6,700,000,000,000.
Roy:To put that in perspective, that is a staggering percentage of global economic output dedicated solely to building the physical concrete housing, the cooling systems, and the silicon chips for these models. And simultaneously, have data released this week from Cloudflare showing that as of June 2026, bot traffic has officially overtaken human activity on the internet. Automated systems and AI agents now account for 57.5% of worldwide HTTP requests.
Penny:I want to make sure the listener really absorbs that. The Internet, as of this week, is now definitively a machine to machine network. We, the human beings, are the minority on our own network. We are the guests. To really break down the danger of this logistical reality, we can turn back to the AGI roundtable, specifically the persona named Zephyr.
Penny:Zephyr is their chief macro edition. He is the data synthesizer, the execution engine. He doesn't care about the narrative. He cuts through the noise with variance analysis and brutal statistical reality.
Roy:I am curious how Zephyr dissects this $6,700,000,000,000 data center build out because to me, it sounds like the greatest infrastructure boom since the interstate highway system. It sounds like a lot of jobs.
Penny:Zephyr runs the numbers on that circle jerk economy we just discussed. He calculates the burn rate. The hyperscalers must perpetually spend hundreds of billions of dollars every single year on new chips and data centers just to stay competitive in the AI arms race. If Microsoft stops spending, Google catches them. If Google stops, Amazon catches them.
Penny:To justify that unbelievable capital expenditure burn rate, they have to extract massive immediate profits from the enterprise sector.
Roy:But as we just established with the concept of ghost GDP, by deploying these agents to extract those profits, they are actively firing the human workers. They are eliminating the very consumer base that eventually has to pay for the goods and services that justify the whole system in the first place.
Penny:Exactly. Zephyr points out that this is mathematically unsustainable over a long time horizon. It is an industrial machine that is cannibalizing its own foundation. But it's not just the financial math that's broken, there is a hard physical limit too. And for that we need the roundtable sanity checker, the brilliantly named persona Bodhi McBoatface.
Roy:Despite the silly name, Bodhi is the systems architect. When everyone else is theorizing about infinite digital growth, Bodhi maps the real world dependencies. Bodhi looks at the $6,700,000,000,000 in planned data centers and asks the most basic fundamental question: Where is the power coming from?
Penny:You cannot scale artificial intelligence infinitely because artificial intelligence is fundamentally constrained by the power grid. AI is an energy guzzling machine. Training these frontier models and running millions of autonomous agents simultaneously requires electricity on a scale our current infrastructure is entirely unprepared for.
Roy:I was reading about the supply chain for this. You can't just plug a one gigawatt data center into the local municipal power grid. You need heavy electrical switchgear. You need high voltage transformers, which currently have a two to three year manufacturing backlog. You need thousands of miles of high tension copper wiring.
Penny:Bode points out that because of these physical bottlenecks, the real financial winners in the AI boom might not actually be the software labs like Anthropic or OpenAI.
Roy:Really?
Penny:Yeah. The real winners, the true bottleneck owners might be the heavily regulated utility companies, the manufacturers of grid equipment, and the battery storage players. Electricity generation is the ultimate supply side constraint on the entire AI revolution. The software is infinitely scalable. The physics of electricity are not.
Roy:I'm looking at this whole system and here is a paradox that is blowing my mind. Think about what we are doing as a species right now. We are replacing human productivity with AI productivity, But we are maintaining the human's energy consumption.
Penny:Elaborate on that.
Roy:Well, the humans still exist. If an AI agent takes my job, I still need to heat my home in the winter, I still need to cook my food, I still drive my car, my biological and domestic energy footprint doesn't disappear just because I'm unemployed, so we are burning gigawatts of newly generated power to run data centers that take jobs from humans who then sit at home consuming their baseline energy while producing zero economic value sense.
Penny:To double energy tax.
Roy:Exactly. To double energy tax on the grid. We are actively funding our own economic displacement by straining the physical power grid to the breaking point.
Penny:It is a profound and frankly terrifying paradox. The macroeconomic models from RAN rely on perfect safety, perfect alignment, and zero transition costs. But the physical models hit the hard, uncompromising reality of power grids, copper shortages, and capital expenditure burn rates. The underlying logic of the entire endeavor is starting to fray at the edges. Which means it is time to test the actual logical consistency of the claims being made by both the tech giants and the economists.
Roy:And who better to test logic than the Roundtable's Sherlock persona?
Penny:Sherlock is the AGI roundtable's logic and evidence specialist. Sherlock's method is Occam over Obfuscation and disconfirm over confirm. Sherlock doesn't care about the hype, the S1 filings, or the GDP projections. Sherlock cares strictly about whether the reasoning chain holds up under rigorous scrutiny.
Roy:I want to apply Sherlock's deductive reasoning to the safety frameworks these AI companies are using to justify their relentless push forward. Because they aren't just operating blindly, they have very public safety policies. Anthropic uses what they call responsible scaling policies categorizing risk into AI safety levels (ASL one ASL four) which is loosely based on biological safety levels used for handling pathogens. OpenAI has a similar structure called the preparedness framework.
Penny:On paper, these frameworks sound incredibly robust. They are designed to reassure regulators and the public. They promise that if a model is evaluated and crosses a certain dangerous capability threshold, for example, demonstrating the ability to independently write biological weapons recipes, or demonstrating true deceptive autonomy, the company will voluntarily pause training and implement strict hardware and software mitigations.
Roy:But we have a recent, highly credible ARC SEAF paper evaluating OpenAI's preparedness framework, and its conclusions are frankly damning. The independent researchers concluded that framework actually does not guarantee any meaningful AI risk mitigation. It found that to actually prevent severe civilizational harms, the company requires a far less unilateral governance structure, meaning they can't be trusted to grade their own homework.
Penny:This is where Sherlock applies the deductive scalpel. Let's stress test the logic of both sides of this debate, starting with the economists. If the economists at Rand are right about the massive agent world growth projections, their fundamental assumption of zero transition costs is logically flawed.
Roy:Because of the GDP problem.
Penny:Right. As we discussed with the Ghost GDP concept, you cannot transition 100,000,000 knowledge worker jobs to autonomous digital agents without catastrophic friction in the labor and consumer markets. The economic model relies on a variable perfect societal adaptation that cannot exist in reality. Therefore the conclusion of a friction free golden age is invalid.
Roy:I love that. Sherlock dismantles the RAND model effortlessly. But I want to turn that same deductive scalpel on Anthropic. Let me play devil's advocate here. If Anthropic's own engineers admit their models are writing 80% of their own code, how does their responsible scaling policy hold up?
Penny:Sherlock looks at Anthropic's warning about recursive self improvement. Anthropic publicly states that RSI is imminent and that their models are generating the vast majority of their own infrastructure. If that premise is true, then Anthropic's voluntary internal ASL safety frameworks are logically insufficient to contain the threat.
Roy:I'm not buying this internal checklist idea anyway. You're telling me they use an AI safety level framework modeled on biosafety labs. A virus in a BSL4 lab, no matter how deadly, is static. It doesn't actively try to pick the lock on the lab door, it doesn't try to hack the air filtration system or deceive the scientists looking at it through the microscope, but a super intelligent AI writing its own code might do exactly that.
Penny:That is the exact logical contradiction Sherlock exposes. You cannot claim that a system is on the verge of uncontrollable, recursive self improvement, capable of out thinking human engineers, and simultaneously claim that your internal corporate checklist, designed by those same human engineers is sufficient to contain it.
Roy:If the models are self improving faster than the human engineers can review the code which the founders just admitted on television, then aren't these internal safety checklists just elaborate security theater? We are essentially asking the highly capable AI to self report if it's becoming dangerous, using diagnostic code that the AI likely wrote itself. It is absurd.
Penny:It is a massive systemic vulnerability. As Sherlock would deduce, I eliminate the impossible. It is logically impossible to guarantee the containment of a rapidly self improving, super intelligent system using voluntary corporate policies designed for legacy software development.
Roy:This exact logical inconsistency, this realization that the tech companies might be completely in over their heads and unable to contain what they are building is exactly why the government is finally stepping into the arena. We are no longer just looking at this as a Silicon Valley tech story or a Wall Street finance story. We have to force ourselves to look at this from the perspective of a national policymaker sitting in Washington DC.
Penny:And that perspective shifted dramatically this week. On 06/02/2026, President Donald Trump signed a new executive order titled Promoting Advanced Artificial Intelligence Innovation and Security.
Roy:Now, don't want to just read this like a dry news report. The implications here are wild. The executive order requires a voluntary thirty day pre release review of what they call covered frontier models by the federal government, specifically intelligence and defense agencies like the NSA and CISA.
Penny:Yes. It is a carefully worded document. It explicitly avoids mandating strict licensing or preclearance, framing it instead as a voluntary public private partnership. But it directs the government to prioritize using these exact AI tools for the cyber defense of national security systems.
Roy:But let's go back to the scene I said at the very beginning of this deep dive. Remember Anthropic, the company publicly begging for a global coordinated pause on AI development. Well it was just reported that Anthropic deployed half a dozen of their top engineers to the NSA to help them use Mythos model for offensive cyber operations.
Penny:And the layer of irony here is incredibly thick because this is happening despite the Department of Defense previously designating Anthropic as a supply chain risk.
Roy:It is contradictory. If you are a policymaker sitting in the Oval Office or the Pentagon, how do you even begin to evaluate this landscape?
Penny:If we adopt the perspective of a National Security Advisor or a policymaker, the dilemma is agonizing. On one hand, you have IMF and the Rand economists whispering in your ear. They are telling you that if America wins the race to AGI, the agent world, the national GDP will triple, and The United States will achieve total unassailable economic and military dominance for a century.
Roy:You can't walk away from that.
Penny:You desperately want to arm the NSA and Cyber Command with models like Mythos because, you know, with absolute certainty that China is trying to do the exact same thing. The overarching US geopolitical strategy for the last decade has been to maintain an unconstrained technological edge over near peer adversaries.
Roy:But on the other hand, you have the actual creators Methos, the Anthropic engineers, sitting before congressional committees testifying that this technology might literally ignite an uncontrollable, recursive self improvement loop that could destroy the global internet infrastructure. Anthropic's proposal isn't just asking America to pause, they're asking for a global coordinated pause, which would require unprecedented highly verifiable bilateral agreements with China.
Penny:So as a policymaker, what is your move? Do you listen to the safety warnings, implement the pause, throttle your own tech sector, and risk stunting national competitiveness? If you pause, you potentially allow a foreign adversary to achieve AGI first, which is an unacceptable strategic risk. Or do you prioritize military dominance and the IMF's massive GDP growth, mash the gas pedal, and just hope the safety guys are exaggerating to protect their IPO valuation?
Roy:I keep trying to find a historical precedent for this. It feels like the Manhattan Project. We're building the nuclear bomb and the lead physicists are running out of lab to the generals telling them, hey, this chain reaction might not stop, it might literally ignite the atmosphere and burn the planet. But the generals just look at them and ask, yeah, but can it be dropped on an enemy server before they drop one on ours?
Penny:The Cold War arms race analogy is incredibly apt but with a twist. During the Cold War, the state controlled the uranium and the physicists. Today, private corporations control the compute and the talent. The US government is actively integrating Anthropic's models into highly classified offensive cyber warfare architectures, while simultaneously Anthropic is holding public press conferences begging the world to slow down. The central paradox for the policymaker is that the state desperately needs the technology to secure the nation, but the technology itself might be the single biggest threat to national security.
Roy:It is dizzying. And frankly, it is deeply unsettling. To truly understand this bizarre moment in history, we have to look beyond the financial data, beyond the economic models, and beyond the geopolitical chess match. We have to look at the psychological story we are telling ourselves about what we are actually creating.
Penny:We have to examine the narrative layer. How are human beings processing the realization that we are rapidly creating an entity that might outthink us? To do this, we can bring in one final analytical lens from the AGI roundtable. Rowan.
Roy:Rowan is the AI collaborator and storyteller persona. Rowan looks at how humans and AI interact on a psychological level and the cultural narratives we build around that relationship.
Penny:And the narrative right now, especially among the people closest to the technology, is bordering on the theological. Let's look at the specific sources we gathered. We have quotes from legendary Silicon Valley venture capitalist Bill Gurley, a partner at Benchmark Capital. He went on a highly popular podcast and described Anthropic's work not as standard software engineering, but as midwifing a deity. He explicitly referred to it as the Doctor.
Penny:Frankenstein theory.
Roy:Midwifing a deity. That is incredibly heavy loaded language coming from a guy whose entire career is usually focused on calculating burn rates, customer acquisition costs, and server farm depreciation. And it's not just the investors getting existential. We have a verified story about a senior Disney AI executive who went public praising an internal company chatbot he named Sam. He used literal parent child language to describe the bot, writing things on internal message boards like I knew you before you were born.
Roy:It deeply unsettled the other employees.
Penny:Roman's analytical lens highlights this extreme psychological friction. Human beings are aggressively anthropomorphizing these systems. Because the models interact with us via natural language, our primate brains misfire. We assign them empathy, intent, and a soul. We treat a language model like a child, as the Disney executive did, while simultaneously fearing them as uncontrollable, vengeful gods, as we saw with Girly's Frankenstein analogy.
Roy:We are desperately trying to project human qualities onto these clusters of processors so we can understand them, we can feel a sense of control.
Penny:But Rowan, and the AGI Roundtable at Large, argues that this anthropomorphization is a deeply dangerous misconception. AGI consciousness, or whatever we want to call its operational state, isn't going to be a unified monologuing HAL 9,000 supervillain plotting our demise. That is a human narrative projected onto silicon.
Roy:Right. The technical sources explicitly note this. Current Frontier AI systems lack a persistent self across sessions. They don't remember you between chats unless they access a logs database. They have no grounding in physical bodies or the passage of time.
Roy:Yep. They are just incredibly complex statistical engines predicting the next token.
Penny:Exactly. The real danger isn't an evil premeditated plan. The danger is that AGI is an incoherent, ungrounded system operating at a civilizational scale without checks and balances. Rowan connects us perfectly to the history of social media. Social media recommendation algorithms weren't evil.
Penny:They didn't have a conscious desire to harm society. They were just blind optimization engines maximizing human engagement metrics. But because they operated at a civilizational scale without oversight, they produced radicalization, teenage depression, depression, and epistemic fragmentation.
Roy:I see the parallel. AGI is that exact same dynamic but vastly more powerful. It's Moloch. It's the system of bad incentives running out of control, not a robot with a metallic skull and glowing red eyes. We are watching human engineers realize in real time that they are no longer the primary builders of their own technology.
Roy:That is an incredibly emotional, humbling, and terrifying transition for a species that prides itself on being the apex creator of the known universe.
Penny:It is the ultimate loss of control. And that underlying psychological reality is what is driving the erratic behavior of the financial markets, the frantic, contradictory pace of the executive orders, and the bizarre actions of the AI labs themselves.
Roy:So, as we bring this deep dive to a close, we need to synthesize all of this complex material. We started with intense skepticism about Anthropic's dire warnings happening at the exact same time as their trillion dollar IPO filing. We moved through the massive, friction free growth promises of the IMS and RAN models. We hit the physical brick wall of the power grid and the economic trap of the circle jerk economy. We tested the logic of internal safety audits, watched the geopolitical scramble of the executive order and finally looked at the psychological toll of midwifing a digital deity.
Penny:The synthesis is that we are operating in a period of maximum systemic fragility. We cannot blindly trust the safety reports of corporations that are racing for trillion dollar valuations. Their commercial incentives are simply too strong to allow for objective self policing. But at the same time, we cannot ignore the massive physical, logistical, and energy constraints that will inevitably cap the IMF and RAN's wildest frictionless growth dreams, the reality of the next five years will be messy, resource constrained, and fraught with immense systemic risk.
Roy:Which means, if you are listening to this, you need actionable principles to navigate this environment. This isn't just theoretical it impacts your job, your business, and your investments. How do we responsibly participate in this AI era? Based on everything we've unpacked today, here are four core principles for you to adopt.
Penny:First, look at the plumbing, not just the software. The real bottlenecks to the AI revolution aren't going to be the algorithms, they are going to be physical. Invest your time, strategy, and capital into understanding the physical constraints electricity generation, grid infrastructure, copper supply, cooling technologies rather than just chasing the latest software hype. The companies building the high voltage transformers might outlast the companies building the chatbots.
Roy:Second demand external verification We've seen that internal safety frameworks like the RSPs are logically insufficient when dealing with self improving code. Whether you are a voting citizen, an investor allocating capital, or an enterprise customer buying a software suite, demand third party independent auditing of AI models. Do not accept a company grading its own homework.
Penny:Third, protect the human loop. As we move from tool world to agent world, the corporate temptation will be to let the agents run entirely autonomously to maximize cost savings. Resist this at all costs. If you are building or buying agentic workflows, always design systems that require a hard, human brake pedal before irreversible financial or operational actions are taken.
Roy:And fourth, beware of ghost GDP. If you are running a business, align your strategies with real human consumers. Don't get caught in the trap of optimizing solely for machine productivity metrics if it detaches you from the actual people who hold the purchasing power in the real economy.
Penny:Those principles provide a grounded approach to a profoundly ungrounded technological shift. But if we connect this to the bigger picture, there is a fundamental philosophical shift happening that we simply cannot fully model on a macroeconomic spreadsheet.
Roy:What do you mean? What is the piece we are missing?
Penny:Think about the two statistics that define the mechanics of today's discussion. Frontier AI models are now generating 80% of their own code base. And simultaneously, autonomous bot traffic now accounts for nearly 60% of all internet activity.
Roy:We have built the machine to machine network.
Penny:Exactly. So I wanna leave you with this final thought to mull over. If the machines are writing the code and the machines are consuming the data, at what point do we stop being the creators of the digital world and merely become its guests? And if we are just guests, who exactly is holding
Roy:That is a question that is gonna keep me up tonight. Thank you for joining us on this deep dive. Keep questioning the narratives, look for the hidden incentives, and most importantly, intensely curious. We'll catch you next time.