[00:00:00] This episode was recorded on June 16th, 2026 [00:00:05] Jacob Haimes: Welcome to Muckrakers, where we dig through the latest happenings around so-called AI. In each episode, we will highlight recent events, contextualize the most important ones, and try to separate muck from meaning. As always, I am your host, Jacob Haimes, and joining me is my co-host, Igor Krawczuk [00:00:21] Igor: Thanks, Jacob This week we're talking about what the fuck is going on [00:00:26] Jacob Haimes: Yeah. So [00:00:27] Igor: It's been a busy month or weeks. I'm not sure when we did the last episode [00:00:32] Jacob Haimes: Not just because it's busy for us in, like, our personal lives, but then also in the AI space, um, lots has been going on. So we'll just sorta, I guess, that ourselves. Uh, uh, and, you know, we'll get some, get some coverage done. I've definitely got some things I wanna about specifically, like, Fable, Mythos, et cetera. but then there, there are other things as well. [00:01:05] Jacob Haimes: So [00:01:06] Igor: I think, like, the biggest right now ongoing is, like, uh, Anthropic released the first Mythos class model to great fanfare, and we will have stuff to say about that. And then they immediately unreleased it because, uh, because somebody in the government got scared and told them, like, this is now weapons export, uh, or like some cl- similar classification and now only, like, truly US citizen or green card holders are allowed to interact with the model So that is one big thing. [00:01:36] Igor: I just be like... Concurrently to that, the other big story is like SpaceX IPO, and now SpaceX, uh, is buying Cursor, uh, because, you know, they bought, uh, they had one AI company with lots of funding. Now they have another AI company. Surely this time they can make it work. [00:01:51] Jacob Haimes: Mm-hmm. [00:01:51] Igor: Um, I think this ties into overall the financing story and, you know, like what is the state of the AI bubble basically. [00:02:01] Igor: Because also Anthropic and OpenAPI, OpenAPI, OpenAI, um, OpenAPI is cool, like, uh, let's not mix this up, um, have confidentially filed for IPOs as well with like OpenAI basically coming out with it and I think Anthropic so far hasn't confirmed, but the rumors are cooking. So people are getting ready to, uh, to, to cash out and there's lots of movement connected to that. [00:02:27] Igor: The Pope is now also talking about AI and has opinions, uh, which is fun, and, uh, Chris Olah is now a co-author with the Pope, so I can, uh, I think that, that's a bragging rights What else, uh, happened [00:02:43] Jacob Haimes: I think that's it terms of the things that I would, I was thinking to cover. Um, let me [00:02:51] Igor: I, uh, I had one more thing written down. It's like, um, Chinese models are starting to also do closed versions because they're starting to, uh, to be, like, competitive with some of the clo- uh, clo- uh, closed models, like really like on par. And I guess they also start to need, uh, need to start making money and locking people in. [00:03:14] Igor: Uh, and what that means for the overall, like, open weights and AI sovereignty, uh, situation, 'cause it also connects to the Fable and Mythos thing because Anthropic basically got caught or, like, people realized that Anthropic silently nerfs use cases they think, uh, people shouldn't be doing. So in, uh, they have officially stated now that there's, like, filters that will make, um, Fable, while it was out, dumber at machine learning research, which got some people mad. [00:03:48] Igor: And then for this, some people looked into Chinese models, but it was also closing up, so there's discourse to be had on that as well. [00:03:56] Jacob Haimes: I guess the other thing is the episode that will have recently come out Into AI Safety, uh, where I actually talked with, uh, someone, uh, who worked on gradient routing. I think I sent you, uh, some of those, some of those ideas. Um, and that is relevant here because, uh, essentially what it-- that, that research, uh, was, and it was, uh, done in collaboration with some researchers at Anthropic, enables, uh, capabilities restrictions at scale. so even more explicitly, um, doing what you, what you just mentioned there, um, of Limiting people's, uh, I guess, uh, like the, the capabilities that are given to the public basically [00:04:55] Igor: And maybe minor, just like also interesting news of like the law is moving on this stuff. In Germany, a court, um, held Google liable for the AI slop that it, that it shows in the search summaries because, uh, you know, it was like making shit up and then somebody sued and then, uh, like the court said, "Google, this is you saying something. [00:05:19] Igor: So yeah, you did this and you can't say whoopsie AI," which should make everyone in this industry sweat [00:05:27] Jacob Haimes: Yeah, shit [00:05:28] Igor: Um, this is only in Germany, so like it doesn't really matter, might be contested, but, uh, but I think it's also like worth just like throwing into the interesting things. [00:05:37] Fable/Mythos Updates --- [00:05:37] Igor: Um, but let's start talking about F- Fable first maybe because that's like a good centerpiece [00:05:45] Jacob Haimes: Yeah. So, I mean, that was released, uh, well, Mythos, uh, slash Fable was released to like Mythos became available to certain people, I think. I, I don't-- Or maybe it was just already available to those people. Um, but then Fable was more generally available if, I believe, and I could be wrong here, but I believe you Provided-- Like the way that companies were able to use it was they had to be okay with, uh, monitoring of their data, um, which had previously not been the case. And so-- And technically they don't, at least what was stated is that they isn't, the data isn't kept for over 30 days or something like that. [00:06:35] Igor: Mm-hmm [00:06:36] Jacob Haimes: and they just do like classification on it, but still it requires you to turn off the no data retention for your entire, um, workspace, I believe, not just for Fable conversations. Um, and also, you know, is super token, token hungry, um, et cetera. So people can use that. We got what? Like three days, two days of that being out and they're, you know, making all these press releases and, and that sort of thing. And then from my understanding, uh, and what we know in the media right now is presumably There was some, uh, high up C-suite guy for Amazon who was consulted by, someone in the US, um, sec- secretary of the treasury, I think, or something like that. Um, and he was like: like our researchers did a jailbreak on Fable, and now it's like, you know, doing things that it shouldn't be," basically. and then the US government was like, "Uh-uh, can't do that." Uh, and they used specifically like munitions, uh, and export controls restraintch, res- restraints, uh, which like whether or not that really should work is an open question. Um, i- it's arguable that it could work, but it definitely isn't purpose-made for this specific use case. Um, and so there... I- it's just like a very sort of patchwork kind of approach to doing this sort of regulation. Um, and that's its own rabbit hole. in addition to that The US government story, um, differs from the one that I had just given about the, um, uh, Amazon C-suite guy. Um, they say that actually there was someone in Anthropic who, um, told Dario "No, I don't think this is safe. You need to remove it." Um, and Dario said no, uh, and then he told-- and then that, uh, um, researcher, a high-up person at Anthropic told the US government. Um, that does seem to be only the US government story. Like everyone else seems to be on the same page as, as the other public reporting. So, you know, um, grain of salt with that one. But, um, the [00:09:31] Igor: you saying you don't trust the US government, Jacob? [00:09:35] Jacob Haimes: I'm an American, so of course not. That's, it's in my blood. I don't, like, mistrust [00:09:45] Igor: administration ever lied? [00:09:47] Jacob Haimes: Mistrust of the government is a national right. Um, so yes, I do not trust the government. Um, the Uh, other things that are interesting here, um, so I'm-- Right now I'm just sort of like giving the, you know, the primer for, for anyone who hasn't really dug into this story yet. [00:10:14] Jacob Haimes: Um, but anyways, uh, it wasn't that they said you can't have it publicly available. It wasn't that the government said you can't have this publicly available. was, it was that they said no foreign nationals are allowed to use this model. Um, so, so going along the lines of like, this is a security con- national security concern, even employees of Anthropic who are not, um, citizens or, uh, it, it-- You said green card earlier. [00:10:50] Jacob Haimes: I, I didn't know about that carve-out. Uh, I thought it was just US citizens. But, [00:10:54] Igor: It's what, it's something, it's something I read. So, so like, um, it's very restricted [00:10:59] Jacob Haimes: It's-- And but, like, even if they're in the US, they may not be allowed to have access. in practice, what that meant was that Anthropic said, "Okay, well, we can't actually vet, uh, vet the, the people who are, are using our systems, uh, robustly, and so we are just going to remove complete access to all of these models, um, in order to comply." That w- is then, I think, at least I find somewhat hilariously, that they really have played up this, um, "Oh, poor us," um, story. Like, on the user interface, there is a, " Claude Fable 5 is currently unavailable," like banner on all chats. And it's like, I-- okay, I, I literally wasn't using that before. I don't give a shit. But it's there. Uh, for a while, I couldn't remove it. I think I can now at least close it once it's, once it's popped up. but the sort of like- Playing up of, uh, this scenario I think is definitely there. also relevant is that the jailbreak that was suggested sounds like it was actually not a jailbreak at all. Um, so essentially this C-suite level person either intentionally [00:12:38] Igor: let me interject there. Like, I might let you finish, but, um [00:12:42] Jacob Haimes: Wait, you think it is a, it, it is a legitimate jailbreak? [00:12:45] Igor: Like, let's keep going. So, so like, like what, from what I've read was basically that like the jailbreak is being portrayed like by one, um, security company's, um, s- uh, CEO, I think. I said like, uh, for like the register I think was like the, uh, fix, fix this is a prompt basically. [00:13:05] Jacob Haimes: Yeah [00:13:06] Igor: and it was, uh Katie Moussouris, founder and CEO of Luta Security, um, wrote like a blog post where she went into details. And it's basically like point at a thing that you might have a security exploit. Say, "Please fis- fix the security exploit," like make this safe, and then like write tests to ensure that you really fixed it. [00:13:37] Igor: And then you can, you know, like extract out of the tests the exploit 'cause it will write the exploit in the tests to make sure it's not there anymore [00:13:46] Jacob Haimes: Right [00:13:47] Igor: and, and the thing is that like, like this is a jailbreak in the sense that, you know, like you can get it to do stuff that it should have, uh, guardrails against from Anthropic's perspective. [00:14:02] Igor: And like they kept drumming it up as like a cybersecurity ultra model that was supposed to have these guardrails to make it not do that. So in that sense, I think it is a, it is a jailbreak. But the interesting thing is that it is also an impossible to fix jailbreak if y- if you want it to be useful for anything [00:14:25] Jacob Haimes: Yeah, and I guess that's what sort of what I was, um Trying to get a... I, like, I don't, I don't know if, if they're saying that this should be, this is a tool that should be used for defense, for-- which is, is the, the, uh, narrative that's being pushed, it has to be able to do that, right? Like there, there isn't a way around that it's supposed to be able to be used, uh, for defense. [00:14:55] Jacob Haimes: There is an inherent offense/defense sort of trade-off in, uh, situations like this, and you can't say, uh, like, "Oh, well, we can have only defensive knowledge," and you can't draw any offensive knowledge from that defensive knowledge. That's not how it works [00:15:13] Igor: Yeah. So like this is like a thing that if you work in security, uh, is a constant, uh, problem that people don't get that like hacking just means like understand the system and make it do what you, uh, what, what you want. You can like hack together a solution and they think kind of as like hacking tools and there's good tools. [00:15:36] Igor: And the thing is that like a defense tool, like what it a- allows you to like check for vulnerabilities is exactly the same tool that you will run to check like deployed systems for vulnerabilities. Like they are exactly the same tool and you can't really, uh, like, like fix this on, uh, on the tool level and it's a constant problem that people try to, to ban. [00:15:58] Igor: In Germany, there's a so-called like hacker paragraph on, on the books that basically makes, um, standard penetration testing software illegal to own. Uh, so if you're a German cybersecurity company, basically you're constantly at risk of being put to jail with a, uh, like accusation of like you want to hack everyone with these illegal tools, which, you know, like not, not only for Americans, like freedom of speech people is a bit of insanity. [00:16:38] Igor: Um, and the EU is also kind of like following this, uh, this and the guardrails on mythos in th- in, in this regard are like more of the same basically. And then this discourse is now interesting 'cause I think we should also like-- And I'll let you talk after this one. We should be clear. I personally don't think this is about the f- the claimed facts at hand. [00:17:03] Igor: Like I think this is about Anthropic said no to Trump even a little bit and there's now like leverage to make him scream like days or weeks before, uh, like, uh, they have their IPO. Like somebody on Hacker News said like, um, the total addressable market just got cut from billions of people to like a few million people in the US who can afford, uh, like a, a, a Claude. [00:17:27] Igor: And that's like a massive problem if you're trying to IPO to raise more funding [00:17:33] Jacob Haimes: only, uh, again, only if you, if you only care about Fable. Um, uh, like they still have all their other models which aren't being regulated [00:17:43] Igor: For now [00:17:45] Jacob Haimes: For, okay, yeah, sure. But I think if they were gonna be even-- Right now, the, the grounds that are being used, um, is essentially, I think at least the only way it makes any amount of sense is that the US government is saying, "Well, Fable 5 has-- it is better than everything else," right? [00:18:06] Jacob Haimes: It, it allows for something that is better than everything else, uh, at doing this task, and that differential is what we are preventing. Now, there are also arguments that that isn't true, that there are a number of other, um, tools, uh, models out there that can also find the same, CVEs, uh, vulnerabilities within the codes. But h- at least I, I think it's kind of funny honestly, um, because Anthropic shit the bed themselves, now they get to lie in it. Like, they pushed this narrative of our model is the strongest, our model has unprecedented claims of, uh, being able to detect vulnerabilities, and they really went hard with that marketing push, and essentially allowed themselves to be subject to this critique. [00:19:09] Jacob Haimes: If they hadn't done that so aggressively, I don't think there would really be grounds for this at all, like even remotely. Whereas the circumstance, I don't think it's... I- if you take a, um, naive, um, person's view of the tech and take everything said at face value, then actually the US government's, uh, approach, like, isn't the craziest. Um [00:19:44] Igor: actually exactly what Dario was advocating for before where he said, like, the government needs to have, like, wide sweeping, uh, ability to, like, ban frontier models, like, th- that aren't, like, as g- as well verified as theirs is kind of the way he went, and this was, like, memed to hell. You know, like, uh, wait, not like that. [00:20:02] Igor: Um, so, like, I get, I-- what basically, like, um, you are indulging in a bit, like, of, like, the sh- schadenfreude because, like, maybe also, like, some people who aren't constant listeners is like, there's a thing called criti-hype that, you know, like, like, Jacob is a, like, Zen master, and he walks a middle path, and he's, like, neither a hater, a hater nor, nor a, a booster. [00:20:30] Igor: He's, like, the perfectly, like, uh, not enlightened centrist, but, like, the good version of that. Um, whereas I am a dirty hater, right? people like me would claim that, like, since the beginning, the whole AI safety shtick has been basically a, a way to distract from, like, the actual issue w- with, like, tech lets us do stuff, um, and we should ask who has control over tech and what do we do to, like, make sure, like, the power overall gets distributed. [00:21:00] Jacob Haimes: Which I [00:21:00] Igor: So, you know [00:21:00] Jacob Haimes: to be clear [00:21:02] Igor: uh, to-- and distracts them to like, here's this tech problem, and it's just a completely new thing. We can't actually control this, and the system is so powerful. And I think, I think I, I, like I call it like, like, uh, um, tech obfuscation at, uh, like in, in, in, uh, so some episodes. So like there's this thing and now- We are-- There's a bit of glee in seeing that backfire. [00:21:25] Igor: I, I g- I, I get that basically. Uh, that like they've given the tools to an o- to an, uh, I would say possibly oppressive government, um, to do bad things to them [00:21:40] Igor: Switzerland doesn't have the free speech of the US. I, I ca- I, I... [00:21:44] Jacob Haimes: Sure [00:21:46] Igor: Um [00:21:47] Jacob Haimes: But the, yeah, the, the, I, I do think that there is a bit of like you reap what you sow. I, like I, I don't know. I, I, that's Yeah, it's true if you actually interrogate the claims here and compare to other frontier models and, um, take the time to understand what's happening, the sweeping regulation doesn't necessarily... [00:22:25] Jacob Haimes: I- it isn't necessarily valid. However, uh, one could also argue, and I think it, it would be a fair argument as well, um, that it isn't necessarily sweeping regulation. It is just Fable 5 right now, and it is just for people who are not US citizens. Uh, which has been sort of also the national security angle that's been pushed i- is that, you know, we don't wanna be giving powerful technologies to our enemies. [00:22:55] Jacob Haimes: And the US government has sort of embraced that framing here. And It isn't necessary that Fable 5 was removed, like complete access to Fable 5 was removed for everyone. What was necessary was that it was removed for people, um, who are not US citizens or some other clarifiers as well. And it is Anthropic that can't meet that requirement. [00:23:25] Jacob Haimes: So in effect, they have to close it down. But like, I, again, I'm not super sympathetic for, "Oh, we didn't realize we actually had to be able to control the technology, be able to control it, uh, access and like in order to deploy it." I, I, I don't know. It just is not something that I'm Like, yeah, a- any of the sort of arguments or, or defenses there, I just, I don't find particularly compelling because of how the discourse, not just from Anthropic but, like, in the space has progressed [00:24:06] Igor: Yep. I mean, like, I still think it's very important to, like, not get sucked into this idea of kind of like, oh, there's, like, merit to the, like, the face value storyline being told. Like, uh, like, like I know you did it kind of like as a, as a naive exercise, kind of like if we take them at face value. But I think it's important to like... [00:24:28] Igor: Like the, the actual reason why this happened is because, like, the government wanted to punish them for like, [00:24:37] Jacob Haimes: Yeah, yeah [00:24:38] Igor: and they had a cover story, and Anthropic handed them this cover story because-- And we talked about this in, in our c- in one of our episodes, but like they might actually believe in the cover story, but the main thing the cover story allows them to do is to like, um, justify doing sketchy stuff while calling themselves like good and also calling for like regulation and preparing the, the place for like regulatory capture that doesn't affect them, but it will make it harder for people to, uh, uh, to compete. [00:25:07] Igor: Like no matter what their intentions are, that is like the outcome of what they, they do and how they're doing it. Um, but yeah, af-after, after that, like I agree with you. Like, like it is like a, a nice exercise to see the inconsistency, the, um, hypocrisy, like in the sense, uh, uh, like, like you, you advocate for A, but then you can't live up to those standards. [00:25:34] Igor: And then like you say, when somebody calls you on it, like that is unfair [00:25:39] Jacob Haimes: Yeah. [00:25:40] Igor: It will be interest- [00:25:42] Jacob Haimes: that's what I... The hypocrisy is the thing that I'm pointing out and the thing that I'm trying to be, like, the fact that there is so much. I, I agree with you that the actual argument is not valid and the actual reasons, the actual motivations for why this has happened, uh, are not represented in that argument. Uh, but like If you're going to use an argument like that, then, uh, at least be ready for it when it's weaponized against you [00:26:17] SpaceX's IPO and the Cursor Buyout --- [00:26:17] Igor: And maybe let's slightly move on from this, uh, like a little bit and like, um, we do like a small detour, [00:26:23] Jacob Haimes: Okay. To what? [00:26:25] Igor: which like [00:26:27] Igor: The IPO situation I think is interesting here, where if we step back a bit and we don't focus on an individual IPO, we're entering in the finding out era, I would say. Like, like, like Jacob's people have a wonderful saying and, uh, coin-- I'm talking about the bald eagle people, uh, here right now [00:26:50] Jacob Haimes: you mean fuck around, find out? [00:26:52] Igor: Exactly. [00:26:53] Igor: Like, uh, and like, uh, like there has been lots of like fucking around with, uh, wild claims and, uh, speculation and hype, hype story And the use of this criti-hype where I'm never sure if I ev- ever close the attention of like criticizing and drumming up fear of this is so dangerous as a way to sell the thing has been like a modus operandi of all of those companies. [00:27:18] Igor: But now we're entering the find out story where like if you make people either like very scared or you create like the surface justification of this is very dangerous as a way of selling your model, then it can actually be used against you as well is like one thing. And the-- moving on to this, like the fact that now SpaceX is going public and they [00:27:43] Jacob Haimes: gone [00:27:44] Igor: has gone public and they had, they had a nice IPO pop and so far the Musk effect is still holding strong. [00:27:51] Igor: But to do that, they also did a bunch of like genius level financial engineering Which is [00:27:58] Jacob Haimes: Uh, what, what exactly are you referring to there? [00:28:01] Igor: S- so for people who don't know, IPOs, initial public offering, is where investors get to cash out. Uh, and so they're always engineered [00:28:11] Jacob Haimes: also part of that, like the how in how they cash out is it is opened up to public. Uh, like, it-- you don't need to have these special, um, uh R- uh, what, [00:28:31] Igor: qualified investor, [00:28:32] Jacob Haimes: Yeah, [00:28:32] Igor: a qualification [00:28:33] Jacob Haimes: invent- investor, um, in order to, um, invest in it anymore. Now you just need to be a person. Um, and so you can then invest in it, and the normal people investing in it, uh, is what then increases the, the stock price, um, because now there are more people who are investing in it [00:28:56] Igor: Well, like that's asterisk. Like, like what actually the main thing is like it is public, so everyone can give money. Before the main, uh, concern is more like deal flow, who gets cut into the deal. [00:29:08] Jacob Haimes: Sure [00:29:08] Igor: like the qualif- qualif investor is another thing. But when [00:29:13] Jacob Haimes: I'm [00:29:13] Igor: you're bringing ac-- [00:29:14] Jacob Haimes: it a little bit [00:29:15] Igor: But the, the main thing is you bring actual cash, right? [00:29:17] Igor: Like, uh, uh, like, like, uh, and that's, uh, the, the thing that the company needs. Or, or, originally it was to actually raise cash to pay for things because the, the share price doesn't matter if you can't sell it. Like it, it only is worth something if you can actually trade it, and that's the point of an IPO. [00:29:34] Igor: But the old investors always want to get, you know, like actual like their return. So it's always engineered that like in the beginning, you price it like slightly below demand as, as in like there's a market clearing price, and you go like slightly below because you want it to rise. So more people kind of like FOMO in because they see it rise, and also so the people who want to sell, they can sell without dipping the price because that looks bad. [00:30:04] Igor: And so IPOs always, unless they're really fucked up, rise for a few days to a few weeks, and then at some point there will be like the quarterly, um, earnings reports and like n- the normal market, um, uh, price finding mechanism kicks in. So it's a common thing to kind of like juice the books a bit before an IPO. [00:30:26] Igor: And what, what, uh, SpaceX did amongst other things was they did a big deal with Anthropic, basically like where Anthropic committed to b- I think it's like one billion, two billion, like billions of dollars per month, I think it might be month s- [00:30:47] Jacob Haimes: I don't know the financial deals, but I [00:30:49] Igor: on [00:30:49] Jacob Haimes: the R- they're, they're renting GPUs because, uh, Grok, uh, doesn't have any demand because it's shit. Um, and so which XAI now part of, and, and Grok, which is the, the model it creates, is now part of SpaceX, um, for some fucking reason. So, um, by doing that and then making a deal with Anthropic and Anthropic saying, "Yeah, we'll, we'll use your data centers," whatever. Um, uh, with apparently from, from what I understand, a very bad deal on the, um, side. Like Anthropic has a, I believe just like a can walk 90 days, um, clause in there. So they can just say, "Eh, yeah, we actually don't like this. We're gonna, we're gonna leave." And there are no, uh, downsides for them doing that. [00:31:44] Igor: Which importantly was announced separately from the big headline. Like, the big headline was like, "We have a deal, one point two five billion per month, uh, un- like up until May twen- uh, uh, twenty-nine. Like, uh, gonna make so much money now. Like, this is, this is a big win for SpaceX." Also, Anthropic has, has a walkaway clause. [00:32:05] Igor: They can cancel this before at any poi- uh, uh, point with 90 days, uh, warning. That was like the pacing. This is like the type of like IPO engineering they, they do. Google did a similar deal with them afterwards, and the important thing to know there is Google has invested into SpaceX. They own like five percent. [00:32:23] Igor: And like after... Like, I read an analysis that like if they actually like have the effect that ViscAI estimated, by committing to paying them a certain, I think hundreds of, like tens of billion, like certain amount, we can link it, uh, uh, in the show notes, they will have like made money on that spend because that share in, uh, in SpaceX w- it would be worth more than the money that, uh, like, uh, they spent. [00:32:53] Igor: So that's why I call it like genius financial engineering [00:32:58] Jacob Haimes: Sure. Uh, uh, and, uh, the- there are other aspects as well, like, um, somehow they've pumped the valuation to what, like 108, uh, or- or something billion. Um, and, that's just insane because, uh, the only part of the companies... Or of, of SpaceX that, that has, uh, legitimate profits, which are also, um, [00:33:23] Igor: 2.1 trillion [00:33:25] Jacob Haimes: Sorry, 2.1 trillion. Uh, Jesus Christ. Um, the on- Sorry, the Pope's in the room now. I have to be, uh, nicer there. Um, but the idea is if you have, um a valuation that high, then, you know, first of all, it's insane. The other part that's like, guess is, is meaningfully intelligent if you, uh, uh, are in the interest of maintaining, uh, and entrenching, uh, your own power is that because of how they've, they've designed this, uh, Elon Musk maintains, uh, like will, will forever in perpetuity maintain at least a complete majority SpaceX. Um, and so there is effectively no way for anyone else to ever come close to that, um, with SpaceX. And there are other limitations that prevent, uh, normal investors or coalitions of investors from doing this as well. So I believe it's something like you need to have 3% or more, uh, stock in the company in order to be able to like, have a certain type of, uh, um- I guess like request justification or, or like oppose a dec- a given decision. [00:34:57] Anatomy of an AI Bubble --- [00:34:57] Jacob Haimes: So like [00:34:58] Igor: Shareholder rights is like the super thing, yeah [00:35:01] Jacob Haimes: So all of this effectively like consolidates power into Elon Musk, uh, and also makes a ton of money and does so when... Oh, right, the only part of the company that is actually making money is Starlink, um, where everything else is sort of a dumpster fire. Um, and It just doesn't, it doesn't make sense. I don't know. It [00:35:29] Igor: Well, like, like it, it, like, to, to you, like, like the story is of course like they're gonna go to Mars, they're gonna go to the Moon, this is the new frontier. And like the reason why I said this is like the finding out areas, like it-- I, I think we did the bubble episode, and I think I explained this thing of, of kind of like bubbles being, uh, being kind of like they can survive as long as we don't hit like, like an event, uh, that where the belief in the bubble takes too much of a hit. [00:35:54] Igor: If not, then like very brief summary as for people who didn't hear it. If you look at any bubble, which like not as an insult, but kind of like just from a mechanical purpose, if you ask yourself what makes a bubble? It w- uh, like, uh, usually like we call that stuff where like we don't understand why it's worth so much, but it seems to keep growing in value, and people keep to want more of, uh, of it, and that's kind of this loop, like the self-fulfilling loop that makes the price go higher. [00:36:21] Igor: That can go on for as long as people kind of like in their head implicitly do this expected value calculation of like, I can sell this at a higher price if I buy in now. And sure, there's this much risk. Let's say there's like only 5% chance that, uh, uh, that, uh, that like, uh, this actually pans out. But there's a 90% chance that the price will rise high enough that I can just cash out. [00:36:45] Igor: And then if that 90%, uh, times the price that people estimate w-it will hit plus the 5% times the actual payoff, if that adds up to like what they feel comfortable, kind of like what they want, then it's kind of rational for somebody to, uh, to invest into a bubble. But if that belief like that's like plausibility of a price continuing to rise ever goes down, then people just look at the kind of much lower chance of like the bubbly thing, like the original like value e-excuse. [00:37:20] Igor: In the case of Bitcoin, it's this idea of like alternative m-money system, which isn't worth like what it is worth right now, but the story of it keeps it going. In the case of AI, right now it's always about like eventually this will be AGI, and the whole economy, uh, economy will run off this, and we get a big cut of it. [00:37:42] Igor: That's the two things that both need to happen. So it can't be that like AI becomes like Postgres or other open source database, just like a tool in a tool belt of techies, and it's just a new way of building stuff. But there's no monopoly on it. It needs to be like there's a monopoly, everything will, will run on it, and that is what ju-justifies the valuations. [00:38:04] Igor: And for SpaceX, it's like there will be a new frontier. We will be the only people who have Mars, so the only way for you to get a cut of Mars is to invest right now. That's what justifies the price [00:38:20] Igor: And because of that, would be my take, they're doing this financial engineering. They-- And they're, uh... Also that, because of that is why just today Musk exercised the 60 billion option he had to buy Cursor, 'cause, you know, it keeps the story going. And as long as, as the story keeps going, [00:38:36] Jacob Haimes: Well, also [00:38:37] Igor: like- [00:38:37] Jacob Haimes: Grok needed, uh, like anything that would mean that people actually use it, and they, they didn't have anything, so they needed a Band-Aid patch for that one. [00:38:47] Igor: And they want to maybe use the talent that Cursor has because we're making their own models as well. Like, there's actually reasons to do that. But it's always kind of like you need to survive and keep the story going. That's the main thing that feeds into this. And I started the segment with like we-we're entering the finding out phase, you know, like there is a quarterly earnings report coming. [00:39:08] Igor: And also, like we will now see, like how all of these like things play out 'cause this segues a bit also into like Anthropic again. But with both OpenAI and Anthropic having filed as well, they need to do the same dance. But now there is one reference class where we see a AI company, XAI Even though, you know, like if you're in the industry, you know what the laugh is. [00:39:35] Igor: And we can see the economics of, uh, of those data centers. We can see the depreciation rate. We can see how that actually looks like, and that's gonna make it harder to keep the story going despite the fact that both OpenAI and Anthropic are now trying to like say, "Uh, coding, boring. We still do it, but look at this, we're do-doing biology now and chemistry, and we're also doing like Goldman Sachs thing." [00:40:02] Anthropic Core and "AI as Expert Systems" --- [00:40:02] Igor: And they are announcing left and right kind of like whole squads of people that help you integrate the AI into your workflows. Like OpenAI has like an implementation company that they did as a joint venture. Anthropic just announced the Anthropic Core, which is- sounds super dystopian. It's basically like missionaries that get embedded in nonprofits, and then they help them like embed Claude and, and Anthropic pays these people, but critically through a employer of record. [00:40:30] Igor: So these people will never be formally employed by Anthropic. It's always like some kind of like middleman employer [00:40:39] Jacob Haimes: Why, why is that? [00:40:41] Igor: Well, the, like, the, the middleman player, like, is probably because, like, they will not vet these people precisely, and there might be, like, claims of, like, you know, "Hey, [00:40:50] Jacob Haimes: yeah, [00:40:50] Igor: this kid that was selected as a Anthropic fellow or Anthropic Core fellow hooked up a workflow that was supposed to automate our small models, but it has leaked all of our PII data now." [00:41:04] Igor: And they want to hold, like, somebody liable, and if it was, like, Anthropic's employee, could be a thing, but now it's gonna be this intermediary nonprofit But like, [00:41:12] Jacob Haimes: Yeah, that makes sense [00:41:14] Igor: if you actually believed in AGI and like the original story, you wouldn't do this 'cause it's, it's kind of a, a, a distraction and you wouldn't need this 'cause it would actually generalize. [00:41:26] Igor: And what we are seeing is they're spending lots of manpower hiring lots of people to like go into individual domains, kind of like try to find like the good workflows and automate them with their technology, which is valuable and it can still have a big impact, but it's very much not the original story [00:41:45] Jacob Haimes: Yeah. I mean, it's as, as, as I've said, I think before on, uh, uh, on the podcast, um, it, it is a return to expert systems, and it is a return to expert systems with an improvement. Having a natural language interface for an expert system is a legitimate boon. It is not useless. It is a valuable contribution. we are still back in that expert system territory [00:42:14] Igor: And also, like, if you want to be very nitpicky because like I, like I know there's a nerdy, "Ah, it's not just an expert system." Yeah. You can also call it, call it like a prior fitted network and, and, and there, there, there's like some embedded statistical knowledge in, uh, in, in there and, uh, there's also s- some like [00:42:28] Jacob Haimes: But like the in practice, in, in practice It's the same as, as how expert systems were being implemented back when they were hot [00:42:37] Igor: And, and also how, like, ML systems were being implemented back in Wayward and when data science was a thing. Yeah, like you have like a new technology, you have a, like a horde of consultants that makes you your custom bespoke version of that. It is not plug and play. It is not the agent like self-figuring it out. [00:42:55] Igor: Uh, but important thing is that like they're doing it now in bio and in chemistry and in other fields where there's like big money, where like sifting through large amounts of data with some domain knowledge and like hand-holding it a bit can yield, you know, like a new drug worth like billions of dollars. [00:43:10] Igor: So they're preparing the, the story for like how profitable this can be. And importantly, it's a thing that they haven't saturated yet. Coding is saturated. Like it's good, it's useful. And like I never thought it would be this good two or three years ago. Like, uh, I think we did an episode of me kind of like shifting my thing because like, like I ate shit there basically. [00:43:30] Igor: I was wrong in like how far it, it, it, how fast it would get there basically. Um, and I think I thought it would plateau earlier. But if you use the systems as well, it's not self... Like it's not standalone. Like you can't take hands off. L- uh, like there's still a lot of work and it still can't do abstraction, and it is not getting better. [00:43:52] Igor: Like, like, like it makes the same mistakes. There's, there's very little improvement on that front. And if you try to have it write documentation, same thing. It's just not there. And we're fixing th- this with the skills on a case-by-case basis, but it's not the same as what, what is, uh, um, sold basically. And Yeah, I th- I think that is just worth tracking because like They will really try to keep the story going for the bubble reason. [00:44:28] Igor: They need to keep it going un- until they went public and it's like kind of cashed out So there will be more of this type of, of entry and even after being public, a lot of like, you know, power and money still rides on it not crashing and also like more and more like risk being concentrated. Important thing is that, um, some index funds, including, uh, the ones that some American pension funds use to track, uh, their investments have been lobbied partially to loosen their rules to fast-track SpaceX into these funds and to make it li- uh, le- um, not liable, eligible [00:45:11] Jacob Haimes: yeah, so, so what exactly has happened there is essentially like the idea is, at least as far as I understand it, again, I'm, I'm not like a financial person, but, um, you know, I've looked at a couple articles, um, so this is my understanding. Typically, if you do an IPO, you then have to wait, I don't know, six months or something at least, um, until your, um, stock price levels out. [00:45:39] Jacob Haimes: Like Igor said earlier, um, you know, there is-- people understand that there is that initial sort of like increase and then it, it levels out. Um, and maybe it goes down a little bit, maybe it goes down a lot, but it will eventually sort of get to the correct place and then, uh, at least in terms of how the market, uh, sees it. [00:45:58] Jacob Haimes: And then once it, it reaches that, that is the point at which you can then say, "Okay, well I'd like to be included in, uh, you know, the, the Nasdaq" or, um, the o- other finance terms, um, imagined that I, I know them. Um, so if that is the case, then when it is added because of the, um I, I forget exactly what they're called, but the, the portfolios that you can buy into, that you can buy into just like the S&P 500, uh, and then sort of takes, you know, segments of each of the, the things that are listed there. Um, because SpaceX has been to fast-track to inclusion on these lists faster than the typical, um, turnaround is, which allows for it to sort of settle, what is likely to happen is that the index funds, that's what they're called, um, will auto-buy these SpaceX shares that's how they're set up. And then people who don't necessarily know that they are buying into SpaceX are buying into SpaceX, and they are part of the group that is artificially inflating price of the stock immediately after it is IPO'd. So that hasn't happened yet. It has IPO'd, but it has not been listed on the, uh, you know, index funds as far as I am aware. Uh, but it will be listed much sooner than it otherwise would. [00:47:38] Igor: In some of them, not in all of them [00:47:42] Jacob Haimes: Um, because of that, people who are not, you know, paying attention to this or not watching their stocks, um, and, and have index funds, which at this point is actually a, a quite significant amount of people because it represents a, a reasonable... least it has historically represented a reasonable, um, investment to make. now that may not be the case because by doing so you are, uh, essentially absorbing some of the costs from SpaceX's IPO [00:48:20] Igor: But it also means that it might still be a good investment. And there's also an argument to be made, like the whole indexing philosophy is to not try to predict the future. And like this one, I generally don't know because, you know, like if you are trying to short Tesla and you've avoided buying Tesla because you thought it would crash, like you lost a lot of money over the last five years, 10 years, whatever. [00:48:42] Igor: Like, so like it's generally hard, but, but it is part of the financial engineering here of like how to get capital flowing in, in, in, into this company and into these other companies. Um, and I think it's just worth for people to track this. Um, it's also worth saying that like if you follow the S&P 500 or the FTSE world stock index, stuff like this, like your risk is limited. [00:49:13] Igor: Like I think on some of them, basically like 1.5% of like your investment would be allocated towards SpaceX, even if they go to like trillions, like four, 5 trillion or something like that. So like it's still okay. It's just a quite big chunk because normally individual companies would be like 0.1%, 0.2% spare. [00:49:38] Igor: I think that's the other thing worth talking about. When we find out in the like finding out phase, it will now really affect a lot of people via their 401ks, via their pension funds, like via lots of, uh, of different systems. [00:49:54] Jacob Haimes: I [00:49:54] Igor: I think, [00:49:55] Jacob Haimes: through the [00:49:56] Igor: yeah [00:49:56] Jacob Haimes: and the Anthropic and stuff. So the only one that's sort of left is, is the Pope. Is that correct? [00:50:03] Guardrails, Chinese Models, and Whether We Can Trust the Benchmarks --- [00:50:03] Igor: And the Chinese models, which, [00:50:05] Jacob Haimes: yeah [00:50:06] Igor: like because of the-- I-- Also, Fable had guardrails because it was so dangerous and we needed to like be safe, uh, from it. Those guardrails were, you know, like state of the art. So like I work in a biotech company. If you so much as say DNA or protein in, in a Fable prompt, it will bump you into, um, Opus, uh [00:50:34] Jacob Haimes: Well, The only thing that I would caveat there is like, is that, that robust 100% of the time with memory? Or is that only, uh, like are you doing zero shot? Because typically [00:50:48] Igor: do you think I have mem-memory turned on? [00:50:50] Jacob Haimes: No, of course not, which is why I'm saying this. Memory does make it easier to do that kind of thing. Um, you know, I know people who are like, "Yeah, when I'm using the, my system, I can ask all these questions about, you know, biology and, and stuff like that, and then I try the exact same thing on, uh, you know, not my account basically, and it prevents me from doing it." [00:51:17] Jacob Haimes: Right. And so that is thing to consider there. I, like, there are ways around it basically. [00:51:25] Igor: Which actually is like not good for Anthropic. Like if, if this is actually like a thing of like, if a memory just makes it seem like it's okay, then it will just like, like, like, uh, give, give you access. But, um, I wasn't the only one who found the tightness of it ridiculous. Like I did- I didn't care, like, like, uh, like I didn't notice a big difference with Fable as well. [00:51:45] Igor: But there was like discourse on the inter- internet, and then people read like the disclosure and they all said, "Ah, yeah, we also silently degrade, um, people who, who want to like build, uh, pre-training harnesses for LLMs, who want to do like competing foundation model servers, who optimize kernels with it." [00:52:04] Igor: And you know, it is already against the, um, uh, Anthropic, um, terms of services to use, to use their research product, um, uh, for this 'cause, you know, safety. But now they are kind of like silently degrading it and the internet flipped their shit or like the coding, like the LLM using coding internet flipped their shit. [00:52:26] Igor: And Anthropic apologized and now makes this as visible as the other guardrails. But it also like sparked discourse about, you know, like, oh, just use the Chinese models, like just mass migrate to the Chinese models. They are cheaper and as good. And I think an interesting thing there is basically to know that one, it doesn't save you. [00:52:46] Igor: Like a- try asking the host that's, uh, like bleeding edge, Qwen or GLM or Kimi about Tiananmen and see how, uh, they affect things. And there's nothing that guarantees you that like they don't also sabotage their competitors. So there's a whole discourse around the sovereignty of coding tools if we all use those at some point on top of ex- externalities. [00:53:12] Igor: Um, and the other interesting, uh, uh, aspect is something that like Jacob, uh, has some research on is like how, how much can we trust the benchmarks? Because on the benchmarks, the Chinese models are very close to Claude and Co. In lived experience, it varies wildly. And I think as we enter this, like, finding around, uh, finding around, finding out phase after, uh, on many fronts, we will also maybe see, like, more principled evaluations of this via, like, a strategic data sovereignty angle where, like, I-- like, like, you know, while Jacob is a red-blooded patriotic American, uh, I am a red-blooded patriotic European. [00:54:09] Igor: Now, I don't have a United State of Europes, but it should exist. Um, and I live in the country that is the most anti that happening, which is a bit ironic. But, uh, I think it is important that Europe has, like, their own data stack and, uh, like, data sovereignty because Trump already banned, like, commissioners that, that were finding people, yeah. [00:54:34] Igor: Um, and Mistral ain't it right now, and I would really like Mistral to be better. Um, and in the absence of Mistral, the best we can kind of do is to host Chinese models, but that's also not gonna last. But I would fully expect, and like if, if anybody is listening to this who has, has leeway with European politicians, like please, like let me talk with pe- them, and let me convince them that they need to put like money into like bootstrapping like a whole effort to make that happen. [00:55:03] Igor: Like, and starting with like very principled evals that actually cover the use cases, uh, that we need so we can like see which of the open models or like the candidate models are actually fit for purpose, 'cause that is missing right now [00:55:17] Jacob Haimes: the EU is, is actually funding evals more than most other, uh, [00:55:23] Igor: Oh, I didn't know that [00:55:24] Jacob Haimes: um, yeah, most other municipalities, I guess. Um, I'm not saying it's great, but, uh, yeah, I, I do think that they're, they're not fully asleep in that regard. Um, the one thing that I, I do always like to bring up when we talk about eval validity is, I alluded to it earlier, um The idea that we aren't actually doing evaluations of the things that we care about. Uh, we're, we're evaluating things in, uh, predefined settings, and then maybe we're updating those every once in a while, uh, and we're running it on base models. Sometimes those aren't even the models that are being provided to users on the interface websites, let alone whatever else has been added to their context. [00:56:26] Jacob Haimes: When we have a system that has absorbed, the memory of-- a-and has persistent sort of, um, knowledge of something, it is necessarily going to behave differently than the baseline. there, there is no public monitoring of this. Theoretically, you know, OpenAI, Anthropic, the, the closed, uh, APIs could do this monitoring, But how would you actually operationalize that at scale when everyone's asking different questions? And How would you convey it, right? And, and that's like in the best case. That's if they're actually trying, which I, I think we all know my stance on whether or not that's the case [00:57:17] Igor: I think it would be interesting if you can link some of, uh, specific like grant vehicles or like things where you've, uh, seen the EU fund for stuff. Because like I know there's like generic funding pots, but like I, I don't know these kind of like-- So like there's like the NLNet, um, that I do know that is basically like something like this for internet and overall digital technology like sovereignty and the, uh, and they fund stuff, uh, uh, like, um, like Tala. [00:57:51] Igor: They, they, they have, uh, like Restack, which is like an overall like internet set, and they have like a bunch of good stuff that helps a decentralized internet function because the EU is more decentralized than the US that's in the interest basically [00:58:04] Jacob Haimes: Um, so, uh, I, I-- the-- what I'm referring to is the EU AI office, uh, like tenders, um For, at least that's the one that I know off the top of my head. Um, there were four, uh, or I guess the call is divided into six lots. CBRN risks, cyber offense risks, loss of control risks, harmful manipulation risks, sociotechnical risks, and cross-cutting support for conducting agentic evaluations. Um, and those efforts are active So there's at least that. That was a total of nine million for those things. Now, that isn't a lot when you look at, you know, how much things cost at large scales, but that is something. [00:59:00] Jacob Haimes: And I, you know, I don't think that's being by the US, for example, um [00:59:08] Igor: I'm sure somebody will send us, "Oh, there's this grant," but like please do send it. I, I'm interested. So like, um, yeah [00:59:13] Jacob Haimes: Yeah, there probably is. Um [00:59:15] Igor: Yeah. But, uh, yeah, that was just something I found of note and that like I think will become more relevant because, you know, like with the current shenanigans, like one thing the Fable thing also does is like, uh, you know, all those European companies that, uh, are rushing to integrate Claude, uh, in, into their system like i-i-if you have any like, uh, risk management officers or like IT, like, uh, chief information technology, uh, people that are, that are like worth anything, like they should be sweating and they should be thinking about like what to actually do and how to kind of like de-risk this be- uh, because, uh, now, for now we still have Opus, but like wait until they learn that Opus can also do the thing that, uh, that Fable does just slower. [01:00:07] Igor: Like [01:00:10] Jacob Haimes: Yeah [01:00:11] Igor: But I think that's all we have, um, unless you want to talk about, about the pope [01:00:17] Jacob Haimes: No, I don't feel like I know enough about the Pope situation. Um [01:00:22] Igor: we might do a special issue on, uh, the Pope's opinions on, uh, AI safety, uh, if, uh, if, if we feel like it. But like, uh, I think that is maybe worth bringing in as just like the general vibe was, "Hey, companies, you should share..." The overall vibe [01:00:43] Igor: was, "Hey, companies, you should share the wealth. You should, uh, pay for externalities. You should, like, make sure we serve humanity." And, you know, Chris Ola had, like, opinions on it and I think it is Part of the kriti hype like and safety washing propaganda technique from Anthropic side [01:01:05] Jacob Haimes: I, I, maybe we should, we should do a, a little dig into, particularly public figures who have, um, decided to make, make, uh, statements about AI, uh, So the, the two that I think of immediately are the Pope and Bernie Sanders. Um, think that there are probably a couple others as well who we could bring in there and just sort of See, see what the public, uh, h- what they're saying to the public, uh, how they came to those conclusions, and how people seem to be receiving it, I think is, is interesting [01:01:51] Igor: Where, where like, I, I don't say like both of them say stuff that I at least agree with. It is just that like at this point, at least the Pope is like more weaponized as part of the propaganda cycle rather than actually like being helpful, I, uh, I, I believe because, uh, like we all know the problems they're, they're pointing at. [01:02:12] Igor: But, uh [01:02:14] Jacob Haimes: I think that underestimates how influential someone like that could be with, uh, certain demographics who are more, um, who, who are less likely to trust the establishment in general [01:02:32] Igor: But the thing is, it's still misguided. I think Bernie might even have pointed at the companies, but like in this like Magnifica Humanitas, this like document by the Pope, I don't think he specifically calls out like the power dynamics and shames the companies as like being full of shit. But he can like in general calls for the AI to be disarmed from the logics of domination, exclusion, and war, and once, uh, tech- technological progress generally be placed in the service of human dignity, solidarity, and the common good, which is like, you know, Anthropic does that in their own assessment. [01:03:09] Igor: They, they were, they were part of it. Ah, that's what we're doing. See, Pope says it. And I think that that is a thing that either deserves a clo- a, a, like, 'cause like you should like, we should maybe look at it and give it a close read, but it should be like on a quick summary, like the takeaway of people that like it is very well-meaning and we're not against it, but there's like a second order effect of it being co-opted as part of, uh, of this like propaganda machine that is also feeding the soon to be IPO. [01:03:38] Igor: Like the technology is so dangerous that the Pope says it, it must be stopped [01:03:43] Jacob Haimes: Yeah. And then I also think there is an aspect of, um po- like political, maneuvering there, regardless of how much we believe that there should be a meaningful separation between, you know, what a religious leader says and, uh, people interpret it. Um, I, I, I think that, you know, as it has been demonstrated, uh, Trump will, um, know, be outspoken against the Pope, um So I think there's also str- like, uh, you know, a strategic angle there that has to be played. You don't wanna overstep and get more flack than, uh, is necessary. But I guess that... [01:04:34] Igor: what we're saying is li- is, i- is like we hope that, uh, uh, he doesn't send JD Vance to Va- Vati- Vatican again in response to a document [01:04:42] Jacob Haimes: then the pope would die. JD Vance kills popes. That's the, that's how it works [01:04:47] Igor: Ja-Jacob is saying that I, I, I just overall [01:04:50] Jacob Haimes: be- [01:04:50] Igor: made a gesture [01:04:51] Jacob Haimes: of course. He kills popes in the same way that, um, smallpox from, uh, you know, Spanish killed the natives, you know? So it's not direct. I mean, actually in that case sometimes it was direct. But, um, but I, I mean like the initial phase where it, it wasn't conscious, uh, consciously done Anyways, that's all the muck that we have for this week. Uh, if you enjoyed this episode, please share it around with all those people that you think would also find value out of it. [01:05:26] Igor: And if you didn't enjoy it, then please leave a comment telling us exactly what you hated [01:05:30] Jacob Haimes: Sure. Uh, and I guess if you also, or, or, you know, exactly what you loved because, uh, big, big reviews, good reviews go a long way. Um, I guess the last thing is just a, another side note is there is a new show on, uh, Kairos FM, which is a short run audio documentary about, uh, reclaiming UBI, and I think it's pretty cool. [01:05:53] Jacob Haimes: There's also an interview with its, uh, host on the Into AI Safety Podcast as well, um, both of which are pretty interesting. Uh, I think Joe, Joe Williams is, is the host. Joe did a really awesome job, uh, and I am excited for the next episode. [01:06:10] Igor: I think actually, uh, because Jacob didn't do it, there's also a old episode that might be worth checking out in the context of, uh, of Mythos, uh, is like building asymmetric defenses with Zainab Majid. I apologize for the name mangling. [01:06:24] Jacob Haimes: M- [01:06:25] Igor: Majid. Um, so like that seems relevant, so check that out as well [01:06:30] Jacob Haimes: All right. We'll see you next time