You can't escape thermal fluctuations. They just inevitably become
significant. So in some sense, like, extropic is a little
Why don't we just create physics-based computing systems that harness
the noise from environments? To us, from first principles of
mathematics, information theory, probability theory, physics, thermodynamics, this
is the future. Hopefully this podcast is like the beginning of a new revolution.
Alrighty, everybody. Welcome to an emergency episode of the First Principles
podcast. We're coming to you on a Sunday
night because we need to understand what the heck Extropic
is building. They've just launched their light paper. Not a white paper, but
a light paper. It is a great introduction to what they're doing and
I've tried my best to dive into it, but I'm actually sort of at this perfect you
know, in between points. Sometimes I know, like, basically everything about what a
company's building before they come on the podcast. In this case, I have lots of questions still.
I actually don't totally understand what the hell you all are building. So
I'm excited to learn. I'm excited for everybody to watch me learn. And I'm just going to throw as many dumb
questions as I can out there. So welcome to the show, guys.
Thanks so much, Christian. Excited to be here. Love your shows. Couldn't be
more excited to share more with the world today about what we've been building
sort of in secret. This is just the beginning of us getting people excited about this
thermodynamic paradigm of computing. And hopefully this podcast is like the
beginning of a new revolution. I'm sure some of your listeners probably
have seen one of my identities. flowing about
online during daytime. I am Guillaume Verdon. I'm
formerly a research scientist in quantum computing, co-founded the TensorFlow Quantum
Project with Trevor here. Back when we were in school,
used to be a theoretical physicist at Perimeter, ended up being a pioneer of
quantum machine learning, which is a field where You use
quantum computers to do a form of physics-based AI
to understand quantum mechanical matter around us, which
is, you know, that's my previous life. Now, essentially,
I'm founder and CEO of Xtropic, and started
this new physics-based computing paradigm and also happened to
run a little thing called EAC to some extent, as
much as one can run it or be very involved since the
beginning. And that's a techno-optimist movement. And
that's sort of the dual identities that many people are
familiar with. But I'll let Trevor
give more of a bio here. I think people are pretty familiar with my
Yeah, I'm definitely not as online as you. Basically,
I'm an engineer who got swept into Guillaume's field. No,
so I met Guillaume back when I was doing my mechanical engineering degree at
Waterloo, which is clearly the best engineering school in North America.
We love our Waterloo interns. I was doing a mechanical engineering
degree. I mostly did manufacturing kind of stuff before
I met him. I worked at a little company called Formlabs, did some stuff
with linear motors. Then I met Guillaume, and he was like, Trevor, do you want
to work on quantum machine learning? And I was like, I have no idea what that is, but
it sounds cool. So that proposal
turned into the Google product that Guillaume was talking about, and
then I got sucked into the quantum hardware physics
and engineering lab down in Santa Barbara. And I did a couple
of years there working on device engineering, modeling,
studying the effect of noise on quantum computers, calibration, control,
pulse sequences, that kind of thing. And after that,
went on to MIT for a bit and got a call from an old friend and
had to come help him out at Xtropic. So yeah, I'm happy to be here. It's been
Absolutely. You guys want to give us the 101, just the highest level. We're
going to dive super deep and feel free in this explanation to
use a bunch of words that people might not know, because then later we'll dive in and try
I mean, essentially, Trevor and I have had
this career trying to build ways to program quantum mechanical
computers, where you try to embed computational tasks into
quantum mechanical physics, right? Quantum is, we're going
to dive into the contrast between quantum and thermo in a sec, but Quantum
is really, you have things in superposition that the physics of the very, very
small and the very, very, very, very cold, ideally as cold as possible,
ideally zero temperature. And there you
get to program the physics of matter,
usually matter or light, and you learn
to embed sort of programs that are parameterized, just
like neural networks. Neural networks have parameters that you train with all
sorts of algorithms that usually use gradients. And that's
kind of where we came from. We brought differentiable programming thinking
to quantum computing. We were very early on on that. There was no software
doing it at the time. And that was our project. And
then in quantum computing hardware,
there you have, the reality is that
you can't cool down a quantum computer to zero temperature, right? And so there's
a mismatch between the program you want to run and the actual physics
of the hardware. The program you want to run runs at zero
temperature, theoretically, and the hardware has finite
temperature. But what does having finite temperature mean? It just means
that things are jiggling. Things are unpredictable. There's entropy. There's uncertainty that
gets injected in your system because your system interacts with the environment,
and we call that noise, right? And so fighting
noise has been the quest to scale quantum computing
so far, and it's been the bane of the
existence of many scientists. So Trevor's background was
sort of at the very lowest level, how you make the
quantum bits dance. Can you filter out noise? Can you deal
with noise? There at the lowest level, I
was more involved at the algorithms and architectures level where
In quantum computing, you try to do a process called quantum error
correction, where you detect errors, detect these
sort of injections of errors, and undo them, right?
And you've got to keep track of how they spread in your computer. The problem
is they're often, by trying to fix the problem, you make it worse. If
the thing trying to fix the problem adds more noise than was there before. And
so, your quantum bits have to be of sufficient
quality, they have to be low enough noise so that it's worth doing
this error correcting process. And this error correcting
process you can view as a form of refrigeration, right? Really,
you're pumping entropy out, you're using energy to pump entropy out
of the system. And so we saw sort of the
road ahead for quantum computing was very long, you
know, reaching the level where you have a very large scale computer where
you're below that threshold of noise where it's worth scaling
up. There's a long road ahead for that. And
we sort of lost patience there. And we're like, well, if you can't beat them, join them,
right? If you can't beat the noise, you should use
it, right? And so we were thinking, Well,
what if we could use the noise? These general AI algorithms, right?
The parent concept is probabilistic machine learning algorithms.
All these algorithms want to be probabilistic, right?
And so they want this sort of entropy and uncertainty present. And
it turns out that even when we run things on digital computers that
are nearly perfect, right? They're deterministic. we
end up sprinkling in noise at sort of a very abstract level in
our software later on, right? Not at the sort of analog
hardware level. And so it seemed like we do all this effort, just
like in quantum computing, we have all this effort to keep things pristine, right?
And in digital computing, You use a lot of power and
energy so that your system is hard to disrupt. The
noise of the environment is trivial compared to the amplitudes of the signals. And
so there, things are very, relatively
to the amplitude of the signals, not so noisy. But then at
the algorithmic level, you add the noise again. So we were like, why don't we just simplify
that and just create physics-based computing
systems that harness the noise from environments
sort of above the sort of temperatures and noise levels of quantum
computers, but noisier and
lower power than deterministic computers. So it's kind of
this in between, right? So we're trying to build a new paradigm of computing from
the middle out in terms of scales. Had to
That's kind of a top-down explanation. There's also a bottom-up
version that's pretty compelling. Yeah, go ahead. If you look at what it takes
to keep making computational devices smaller, what
you find, and it doesn't really matter what the device is, what you find is when
you make it sufficiently small, you can't escape thermal
fluctuations. they just inevitably become significant,
right? And so if we want to keep scaling computes
smaller and smaller, it's actually inevitable that you have to
go into this thermal or probabilistic regime, right?
And this is becoming, you know, if you look at the data for
like digital computer scaling, you can see that the
rate of exponential growth in efficiency of
computing technology is starting to slow down. And
that's because you're starting to hit some of these effects. There's a ton of reasons
why it's hard to make transistors small, but a lot of them come down to
the fact that these thermal fluctuations are starting to get really big. So
I expect within the next several
generations of transistor technology, you're
going to have to start looking at some of the things we are. So in some
sense, Extropic is a little bit inevitable, and
we're just trying to front run the danger
Yeah, that's really interesting, because you've hit on two different types
of computing. The top-down answer kind of came at it from the quantum angle, and
then this bottom-up one that you just answered, those are just normal
digital chips or whatever that we're talking about, just normal digital transistors. I
would love to take this conversation sort of piece by piece. Maybe let's start with
the first paradigm, which is just normal computing. talk about what
are those chips, like how are they, you know, they're getting down to the nanometer,
like single low digit, like two or three nanometer size
now. So let's talk about that. And then let's talk about quantum and then we can kind of use
that to bridge over into thermal. But on the
quantum, so on the classical note then, do you mind just telling us how these
algorithms and, you know, all this, you know, neural network stuff
is run today? Like, what are those chips? Like, what do they look like? How do they work? And
So to start at the very, very low level, right, what
is a transistor? Yeah, exactly. A transistor is
actually many things depending on what kind of voltages you
put into it. But in the regime that digital computers operate today, transistors
are switches, right? And you network these switches together
to do digital logic. And so The mathematical abstract
thing you're trying to do is Boolean logic, and the way
you embed that in physics is by driving transistors
really hard. And so that's how computers today work, right?
So you're taking these kind of inherently fuzzy
devices, right? They're made out of real matter, so they're fuzzy.
And you're applying very large signals to them so that they behave like
this mathematically abstract object of
Boolean logic that you want. Right? So that's kind of
how digital computers work. If you want to run, let's
say, a sampling algorithm on a digital computer,
which is what a lot of probabilistic algorithms come
down to, that's kind of like one of the main subroutines, because
the dynamics of your device are completely deterministic, right?
They're operating in this kind of high signal regime where the natural fluctuations
of nature aren't important. you have to generate pseudo-randomness,
which is instead of harvesting the noise of nature,
you run a circuit that has really complex and uncomputable dynamics,
right? And so you get kind of streams of bits that look random. And
that process takes a lot of entropy, right? Because a random stream
of bits is kind of like heat in the sense of
connection between thermodynamics and information, and you're using electricity
to produce that heat, right? So it's like you're running an electric heater on your
chip, literally, is the analogy to thermodynamics, right? And
then, okay, so now you have a random bit stream that's not computationally useful
unless you happen to want to do coin flips, right? So then what you have
to do is you have to take that random bit stream and essentially filter it to
get samples from the distribution that you're actually working with. right?
And that step of filtration also takes a ton of energy because
now it's like you're taking this bowl of heat you have and you're putting it inside of
a freezer to cool it back down a little bit, right? So the
process of sampling on a digital computer is thermodynamically
pretty similar to running an electric heater inside of a freezer to
achieve some kind of intermediate temperature. So it's a little bit ridiculous, right?
When you look at it from that perspective, it's like, this doesn't make sense, but it's
clear how we got here, right? Because digital computers are really nice and they're
very easy to scale. So it's convenient to do things this way,
but it's from first principles, it's not even close to the best ways.
Yeah. This approach to sampling is like so inefficient on
digital computers that people, unless you're like on a Wall Street
where things are super mission critical and you're willing to throw a ton of compute to
get the best quality sort of uncertainty for
your decision-making, unless you're on Wall Street, you end up trying
to avoid sampling entirely, right? Because it's
too costly on our deterministic devices. Again, as Trevor
mentioned, it's really unnatural for our deterministic devices to be probabilistic. And
so another way, instead of sampling, to represent probability distributions is
usually through deep learning. And what deep learning does is it usually
starts with very sort of trivial randomness, like
a Gaussian blob, a single Gaussian blob, and then it deforms that
blob to shape it into the shape of the data. So it's
a high-dimensional blob, and high-dimensional data could be like images,
text, whatnot. But it has to use many, many transformations
to take that very simple randomness and transform it into the
shape of the data. And very often, that sort
of fails to capture the tail events, the
tail distributions, a low data regime, right? When you're focused
on like covering everything with one blog, essentially, you're
just going to cover sort of the center of mass or like, of
probability mass, like the typical data, right? You'd be focused on
that. And you're going to need like, more and more and more dimensions and
more and more parameters in a deeper and deeper
transformation that's more and more complex. So you need more parameters, more
data, more compute in order to reach in that sort
of low data regime in those tail events that are very rare,
right? And so we've been seeing that with sort of self-driving cars.
In self-driving cars, we've just been throwing metric
tons of data at the self-driving problem to
reach a level that a human reaches with like 10 hours
of driving classes, right? And there's clearly way more than 10 hours
of data in all the data sets of all the players. And
so that's sort of fundamentally the reason that
current day deep learning is not
quite the end game. We think that this sort of probabilistic approach
where you can use very little data and you
can fill in the blanks with noise, with entropy and uncertainty, Right?
If you don't know something, you don't have data, you should fill it out with uncertainty.
But that process of sort of painting everything with
a noisy brush is very costly, because you got to sample,
you got to like, you got to explore those parts of landscape, you got
to kind of hallucinate everything that's not data, or,
you know, within your model, within the scope of your model, and sort of penalize the things
that are too far from data. And that sort of process of hallucinating
all these possibilities and making those corrections, for
the technical folks, it's called contrastive learning. That
process usually requires sampling, and that's very costly, so
people avoid it. So they stick to these sort of taking these Gaussians and
deforming them. That's how old school Old school neural
nets like variational autoencoders work. It's somewhat how
diffusion models work. Diffusion models kind of mix in the noise as
you go to some extent, simple noise. But
that's kind of the common pattern essentially. So both
from a sort of hardware standpoint, it's inevitable
that we're gonna have to go stochastic because matter is
jiggly and so your transistors are are technically jiggly,
and so will the electrons hopping across it. And so it's going to
get stochastic. And the algorithms want to go probabilistic to
be more data efficient. And so that's why we're building the whole
stack. And we think it's going to be disruptive for everyone. And that's why
we're really excited to sort of put our thesis out there of
the future of AI, which is very contrary and very different, but if
it succeeds, it changes everything, right? And so, at
least to us, from first principles of mathematics, information theory,
probability theory, physics, thermodynamics, this is the future. And
Yeah, basically. I love
it. So to take just a tiny step back, can you talk about what
is it that makes a GPU so good at
doing that sort of estimation task, basically, of
making it so that you have this really crazy distribution and
GPUs do the deep learning approach, right? Because they suck at
the sampling approach, right? So often people use
CPUs for Monte Carlo sampling because
it's a very serial task. You gotta like have little walkers that travel. You're
simulating a sort of particle in this landscape, whereas we use
literal particles to do that job, right? So
a GPU really got lucky, right? A GPU was not
imagined from first principles to be a processor for
AI. It was a graphics processor that did
really well with matrix multiplications. And
it turns out that, you know, these transformations that I was talking about
to morph a simple distribution into
a complicated one, a lot of those transformations, the
big computational element, are matmuls, matrix
multiplications, right? And so GPUs are accelerators for
that. And so most attempts that you've read
in the news or over the past several years that
have been trying to accelerate software
for AI, they've been focusing on accelerating matrix
multiplication, which first of all, you're competing with NVIDIA. Good
luck with that. Jensen will eat your lunch and thank you for it. But
Trevor, you have some first principles reasons why you think And
from the back of the envelope principle, you know, any sort of matrix multiplication
accelerator has a fundamental bottleneck, and
it's not worth necessarily pushing in that area. It's much more
interesting to try to disrupt how
you do the entire algorithm rather than just a subroutine, right, Trev?
Yeah, so the basic reasoning here is if
you go into PyTorch or something and profile a
neural network like a transformer, right, what you'll
find is that you spend about 25% of your time
moving things in and out of memory, right? So what
that means is if you accelerate the other 75% down
to zero time using your fancy accelerator, maybe some
kind of optical MatMul accelerator, right, that literally does the math of
the speed of light, you still only have a 4x speedup because
you're still paying the 25% of time to move things
in and out of memory, right? So accelerating part
of an algorithm only ever gets you kind of a modest speedup.
And so you do a lot better if you look at tasks that are much more
compute bound, like sampling. So
that was kind of another reason we thought this
Is there a reason that you can't do, so I'm, this is maybe skipping ahead,
we haven't really talked about this yet, but you hinted at it when you said that, hey, this
Gaussian, whatever, this like normal distribution
thing isn't gonna be the answer for the future. Like
you wanna do different types of probability distributions with your
chips, right? And can you talk a little bit about why that is? Like what is
so wrong about this normal distribution? And then why
can't we just do those other distributions with normal, like analog or
Yeah, that's a great question, actually. We
use Gaussian or normal distributions, right? It's basically what
is known as a bell curve. We use those all the
time because we can actually keep track, like
fundamentally, what is a Gaussian? It's like a blob, there's
where is it in space, and then how is it squished along which
axis, right? And by how much, right? So
the squishing is a matrix called covariance matrix, and the position is
called a mean, right? And if you have that vector in
a matrix, you fully specify the distribution. So essentially,
it's a way to cheat and have deterministic computers represent distributions,
because they just need to store a matrix and a vector. And they're they have
a proxy for distribution. And you can sort of analytically for
many simple transformations, keep track of how the
Gaussian gets morphed, right? And these tricks
are actually why diffusion models work so well, right? Diffusion
models, they approximate every transformation as like a
slight transformation of a Gaussian. And
so essentially, it's kind of an artifact of them being
some of the, I mean, obviously the simplest distributions you can come up with. And
essentially being easily representable by a classical computer. If
your computer can natively represent much more complicated distributions, we
wouldn't have that sort of bias, right? And the
problem is, you know, there's distributions that have much longer tails, right?
They're not just so concentrated around one
mode. They have all sorts of, you know,
blobs and long tails where, you
know, a very, you know, very low likelihood event
still has, like, a non-trivial probability, whereas Gaussians, as
you get far away, you know, they get, like, more than exponentially low
probability as you get away from the mode. And so, you
know, many machine learning algorithms and machine learning algorithms are
really good at modeling the typical case Right.
And we feel this with LLMs. Right. They're kind of like basic. Right.
Like they're really good at like typical things, but like, it's like, I
need this sort of like edge case. I
need this sort of edgy thing. Like they can't, they can't go there with
you. Right. But human brains can. Why is that? It's so weird. Right. Like, just
like if we, if you're driving and you
see something that's never been in a dataset on the road, you don't
like glitch out. You like, you generalize. Right. And so.
Fundamentally, it's like the constraints of
the hardware, deterministic hardware has constrained our thinking in
terms of where the algorithms are going and
where they should stay. And that
has sort of held back AI. So something, you
know, our ultimate goal here by proposing new hardware is
to also disrupt how software and AI
works and which algorithms tend to dominate and do well when
But what is it about those algorithms that make it impossible or
impractical to model them using a classical computer? It seems like,
I don't know, when I was a stats monitor, I could do a little binomial
plot, you know what I mean? That's a non-Gaussian distribution. What
I mean, if you try to sample directly from a hundred million dimensional distribution,
right, you know, directly without using
Well, the fundamental reason, right, is like, if I have,
let's not go to a hundred million dimensions. Let's start with one and two.
If I have a one dimensional distribution, right, that's just a
function in 1D. So I can slice that function up
into n chunks and store those n chunks in memory, and now I have a
representation of the distribution. Now I go to two
dimensions. Instead of having n chunks, I have a
grid of n squared chunks. So now I
have to store n squared things in memory to represent the distribution
in generality. What if I go to d dimensions? If
I have n slices along each dimension, I have a d-dimensional hypercube
of things to store in memory, which grows really fast, right? So the
general point here is that the complexity of representing a
totally general probability distribution tends to grow exponentially
in the dimensionality of the system. Right. And so, um,
and obviously there's like a lot of caveats that argument because the
representation, uh, like the complexity of the distribution doesn't
have to be exponential, but it can be. And that's kind of the key thing that makes
this difficult on a classical computer. Um, you
can't store them in memory. So you have to sample and sampling has all of
these problems I discussed earlier. So dude, you're just
And so is this something that, was this thought, like this kind
of train of thinking, was this what led people to want quantum computers in
the first place? It's like we can represent these super high dimensional aspects
of reality by just remaining high dimensional, by
Yes. Yeah. So that was a big, so back
in our days in quantum computing, I would just keep hammering home Don't
use a quantum computer for probabilistic machine learning or classical machine
learning, as we call it, because quantum computers are really good
at quantum interference, not necessarily probabilistic inference. Yes,
you can. It's kind of like using a
rocket that's on, you know, rockets are finicky and
less reliable to ship something across town. It doesn't make sense,
right, intraday, right? It doesn't make sense. It is gonna blow up. You
know, there's a chance it's gonna blow up. Like, why would you do that, right? Sure,
like, in principle, it could go much faster, but, like, there's
a chance it could blow up. So what we've seen is sort of, yes,
on paper, a quantum computer can do slightly better
for probabilistic inference. I've written a bunch of papers on this, because I
wanted to, like, rule it out properly, right? So I've spent the
last eight years, I guess I put out my last paper in this space, a
week ago for fun, because it was on my shelf for two years, but I
thoroughly studied, can you do classical machine learning on a quantum computer? It
seems to me like the main advantage is instead of
having sort of jiggly particles that hop above sort
of barriers in landscape, you can tunnel through. So there's an
advantage if your landscape has very thin barriers, because you
have a form of quantum tunneling, right? So sometimes,
like in very special cases, you can find an
optimum a bit faster, but when you do the whole systems
thinking, the full stack thinking of like, okay, I
have a quantum computer, I'm gonna have an error correcting system that's like 99% of
the computer, 99% plus of the computer is the error correcting system, and
I'm gonna have the cooling, and I'm gonna have the control systems, why the
heck am I going through all that trouble for this tiny speedup, right? So basically, it's
not worth using a quantum computer for these sort of low order polynomial speedup,
these sort of, hey, you know, like, I
get a square root speedup, and it's still slow,
it's still relatively slow, and I have to prop up this huge
computer to do it, when you could do it
just, you know, much cheaper on even a classical digital
computer. And so in our case, instead of trying to seek
sort of asymptotic, what is called asymptotic speed ups, like in quantum
computing, like there's, there's literally different complexity classes,
if you have a quantum mechanical computer versus a classical computer, we're
just, we're not trying to violate any sort of laws of complexity theory, we're
just doing, you know, Classical algorithms, algorithms
that you can simulate theoretically on a classical computer, we're
just doing them way faster by a large, like
sort of constant factor speed up, right? And
that constant factor speed up is several orders of magnitude.
sometimes more than can fit on one hand, depends on the algorithm.
But before we pin down exactly
what those speed ups are in the public, we want to put
out some careful scientific works. So stay
tuned for that. But it's very substantial. It's enough that it's
worth going through this exercise of rebuilding
the whole stack from first principles, right? That seems like a huge change,
right? We're taking a fork in the tech tree. We're forking off
the root node. That seems like a huge effort. Is it worth the payout? We think so,
at least from first principles. And so that's why we're really
excited, you know, and that's why we're kind of, you know, we've been
very secretive. Unfortunately, I got As
we know, I got doxxed in December. The plan was always to reveal more in
March. And so here we are. So it's right on schedule.
But our goal here is for people to be open minded about
the future of AI. I know right now it just feels like the
current labs doing LLMs, that's the end game's future. They've
captured the market. It's over. You either work for one of these companies or
you missed out, right? I don't think so. That's the beauty. of
disruption. That's the beauty of this sort of techno capital acceleration. A
couple of crazy kids, you know, with one
or two GPUs can have an idea that can, you
Yeah, that drive, like the reason behind
that makes a lot of sense. I think that the promise of
quantum computers, at least the way that I understood it, was that eventually they're
going to be so, they're going to get, you know, n squared number
of operations in the same amount of bits or whatever. So
we're going from bits to qubits. And when we have qubits, like
pretty soon we're going to have quantum supremacy because you can see like even
the biggest classical computer will be so much smaller than this puny, you
know, or even this very small quantum computer. But
there are problems. There are things that it's not simply captured in
the number of bits or qubits. There are other considerations that
you have to have when building a quantum computer. And I imagine that you two probably have
very strong opinions about that. So I would love to ask you about that. Maybe Trevor,
There. So for stars, it's funny you mentioned quantum
supremacy. The way we achieved quantum supremacy back in the
We were there. We were there a thousand years ago when it happened. Only
Yeah. Back in the day. The problem
that I have with quantum computing, and the main reason I stopped working
on it, is because most of the phenomenon that
are important to humans do not have long-range quantum
coherent effects. So all atoms are governed by
quantum mechanics, but things that are macroscopically observable, that
involve a large number of atoms, don't
need to be simulated on a quantum computer, right? Like our classical models
of them work really well. And so that's one of the fundamental reasons
it's been difficult to find a practical advantage in
quantum computing, right? Like we have these kind of, you know, there's like
a Shor's factoring algorithm, which is like the most common
thing people tout that it's going to like break RSA and whatever, and it
might, but we can just use a different crypto protocol
that isn't broken by a discrete log and
such. So it's very unclear,
even if we had a big quantum computer, what you would do with it.
And that, to me, kind of made it hard to
dedicate my life to it, right? Because the physics
and engineering challenges involved in building quantum computers are
extremely formidable. And after you do that for
I, you could see that Trevor worked close to the metal where,
you know, the challenges are extremely hard. And, you
know, I was a theorist and an algorithmist, you know,
a bit isolated away from the difficulties. I was aware of it
because I would talk to my neighbors and so on. But, you know, the ideal
thing with quantum computers is that they can represent
and sample from states of very high quantum complexity, right?
So, if you have a state of very high complexity, but
it's still, you could still sample from it with Monte Carlo, you
could just run, again, a Monte Carlo algorithm, maybe it's a million times slower than
doing it by nature, but it's still, you're still going to
get there. You know, you just throw a lot of compute at it, you're going to get there, you
know, it scales linearly. The thing with quantum complexity is
that it scales up in some cases super exponentially, like
in terms of how much classical compute you need to use in
order to replicate that distribution. What was achieved in the
Google quantum supremacy experiment and then later surpassed by
Chinese simulations and then reiterated by Google
quantum, so it's been kind of a little race there, but
essentially it was just sampling from any sort of quantum
program that you can't sample from with a classical
computer, even if you were to throw most compute on earth towards
that end. And that was achieved, I would say,
so I don't think there's anything stopping us from achieving
that. I know it's still controversial, but essentially
the promise there was that, okay, if we can show we can sample
from these complicated distributions, right? The narrative, at
least for the quantum AI side, was that, okay,
well, if we have these classes of distributions, maybe
we could search over that space and represent very highly complex states
in nature with these complicated distributions on our computer, right?
And then map one to the other, and that unlocks the ability for us
to sort of understand matter at a quantum mechanical level.
There, there was a lot of challenges to train such such
distributions because when they get really complicated, they get really
hard to train. It kind of is a sort of conservation of difficulty. So
until the hardware gets much better, it's very hard. for
you to use a quantum computer, even if you're trying
to just generally model nature in sort of native fashion,
right? You're trying to model quantum mechanics of nature with a quantum mechanical computer,
running a quantum mechanical AI representation. It's still difficult because
if the computer's not reliable enough, you can't make it big enough, you can't run
the easily trainable representations, and you're kind of screwed. And
so from the algorithmic standpoint, it was also sort of doomed
in the near term, I'm more of an optimist than Trevor. I think,
you know, humans are really smart. I think on a 20 year time
scale, people are going to figure it out. But again, for us, it's like, okay, we're
trying to do all these applications where it's not clear that
you need quantum complexity, right? Really what you
just want is a computer that allows you to do probabilistic
machine learning and optimization very
cheaply, very energy efficiently, and very fast. And
for us, it's like from first principles of thermodynamics of computing, it's
not going to be a digital computer, it's deterministic, it's not going to be a quantum computer, it's
going to be a thermodynamic computer that achieves that. And so that's what we
got to build. And so that's why we left all the secret labs. You
know, I was at Google Apps working for Sergey and Trevor
was like a different black ops lab in Santa Barbara. And
then we joined forces. We both kind of left that. And
now we're here. And now we have this thesis that
we've kind of kept close to the chest. But, you know, now
we're telling it to the world and we're asking if people want to join us. And
Yeah. And one more point on quantum computing that's interesting
in contrast to what we're doing at Extropic. To build
a quantum computer, you have to build some really weird system at large
scale. So that might be superconducting circuits where you're
making Joseph's injunctions, which are not new, but at
least a relatively new object, you might be doing neutral
atoms where you have to build these big arrays of optical tweezers and tables
and tables of lasers. Trapped ions is very much the same thing.
My point here is that the kind of manufacturing and supply chain
for all of these things is extremely immature. And so there's
going to be decades of challenge just there, achieving scale, right? Versus
if you build a thermodynamic computer, what you need to do that
is a noisy circuit. And I can think of lots of ways
to make noisy circuits that lean heavily on
the way we know how to make circuits today, right? Like the whole semiconductor industry.
So that's ultimately what we're chasing here is something that
we can do, you know, in this decade, not several decades from
Yeah, so that's a perfect tie-in. Let's just hop right in and
start talking about what you guys basically announced in this light paper. I
mean, you mentioned Joseph's injunctions. That appears strongly
in the light paper. We
talked earlier about having to keep quantum computers extremely cold, and
I believe that that also is true of this first wave that you've announced
here too. I don't know, from a first blush, I would imagine some
people would think, well, it sounds kind of familiar. What you just said is like, you know, hard to do.
So what's the value there?
We're starting within our neighborhood and we're taking a path
to sort of room temperature and large scale manufacturability, right? We're going
from the bottom up. We're going from the very cold, using similar building blocks
to what we're used to engineering and quantum computing and operating
them in a thermodynamic regime where there's no more quantum coherence. There's
no superpositions of states anymore. It's just fuzz,
probabilistic fuzz over states. And that's where we're operating the
devices, right? And for us, it was just our native
language. It was the first sort of concept of a programmable and
parametric thermodynamic computer we thought of
building. And that was basically our first prototype. And
for us, there's a lot of learnings there of like, how do you even program this thing?
How do you map all sorts of applications to it? What is programming gonna
look like? How fast can it get, right? And showing
the world, hey, this is how efficient you can have neural
computing, computing for AI be and how fast it can be,
fast and efficient, speed and efficiency. It's
very similar to the Tesla Roadster, super expensive, very
exotic, had to import a bunch of parts from all
sorts of suppliers, wasn't vertically integrated yet. And
then that's a stepping stone towards a large scale mass production, in
our case, eventually room temperature. chip that we're going to build.
And we have a roadmap to that. And so for now, we're
just showcasing the world what's possible. Hey, you
have this new paradigm that's coming, we have a first instance of it, but
we have a roadmap to get to sort of having a thermodynamic
Yeah, like in CMOS. So like, you
know, the same way you make your digital computer that you're likely watching this
on, we know how to make thermodynamic computers using
the same manufacturing technology that operates at room temperature, which
So how do they work? What's the, you know, what's the,
like, can you explain the 101 of what is
Yeah, I mean, let's talk about, let's focus on the superconducting chip,
that's the one we're disclosing, the CMOS stuff, you know, we're keeping
on a high level for now. It's the same concept, a lot of the software maps over
in thinking, but it's, you know, just like in quantum computers, you can have
different substrates, right? There's optical ion trap, you
know, photonic superconducting quantum
computers. There's many ways to do it through a computer. This is
a first way. There's gonna be a better way later, but for now,
we're talking about this one. So this one, Trevor's
going to give you a much better, more technical explanation. But essentially, we're
just using jiggles of electrons that happen in superconductors. In
superconductors, electrons like to bundle up. They like to pair up.
They're called Cooper pairs. And when they pair up, they can pass through each
other. That's why there's sort of no friction, you know, there's no
traffic congestion for your electrons in superconductors. That's why they're
superconducting, right? They have way less resistance. For us,
the superconducting aspect is more to have a
sort of non-linearity in the landscape. So that means not it being
a simple Gaussian, right? So if you have a simple LC circuit like
you do in high school, you know, and you add some, some
noise to it, you're going to get a Gaussian out of it. But we didn't want
that. We wanted programmable. super general, fully general landscape.
Essentially, what we do is something called energy-based
models. I'm more on the software side. Trevor's going to give you more
of the hardware side picture. But energy-based models
are models where you try to model data
distributions as equilibrium states called Boltzmann
distributions of certain parameterized landscapes. So
essentially you shape some hills, right? We
have little knobs that we could tweak and it changes
the shape of some hills and we pour some
sort of, you know, just a bucket of bouncy balls
and keep shaking it as we go, right? And that's it,
right? And then the algorithm is just changing this landscape over
time and the bouncy balls kind of flow. But, you know, on average, the
probability mass of where your bouncy balls are. kind of changes and
we guide those bouncy balls. For us, the bouncy balls are literally
electrons, but theoretically, you
can make it out of all sorts of other stuff. But in
our case, that's it. Essentially, we have a programmable probabilistic
computer that has parameters that you can train in
order to morph this sort of equilibrium distribution of
the bouncy balls by morphing the sort
of landscape in which they're dancing. And
we have algorithms that are physics-based to tune that sort of landscape
that correspond to machine learning, you know, like cross-entropy
minimization, which is what transformers do and diffusion models do,
amongst others. And so essentially there's a
connection between, you know, machine learning really is operationalizing
information theory, information theory and entropy, right? The
theory of entropy from Claude Shannon. appears in thermodynamics as
well, right? So we're instantiating information theory as
thermodynamic processes. And so that's the bridge between machine learning and
Trevor, do you want to go, Trev? Yeah. Sure. I
mean, actually, you kind of have absorbed my talking points at
this point, so that was pretty close to what I
would say. I'll add a layer of
generality. In a sense, any circuit you build experiences
thermal noise, so that the charges that are moving around your circuit
are getting battered around by vibrating atoms. So
every circuit you build works that way. The
trick in designing something that's not
just kind of noisy, but very noisy, is
that you have to make sure that the noise is
significant compared to the other energy scales in
your device. Right, so that's kind of where the device physics
and hardware engineering, more hardcore stuff
kind of comes in. But once you figure out how to get into
that regime, basically all you need is some kind of circuit component
that's tunable, that lets you kind of change where
the electrons prefer to sit. And that
gives you a programmable sampling machine,
basically, right? So the principles at play here are
pretty generic and you can imagine a ton of different ways
to build it. And so we're just kind of thinking like, well,
what's the most scalable thing we have? semiconductors.
So basically, I think the thing that is still confusing to me is like,
what's the input and what's the output? So the input, as I
understand it, is you're giving like weights or whatever to each
of these things, each of these, what would you
I mean, it's like a neural network, right? You have inputs, like data, and
then you might have outputs, and then you have parameters. And those are
So you have the parameters which you input, which are the ways
To be more concrete, I think that'll be helpful. You could think of
like data and parameters as voltages, right?
So I apply some voltage to the circuit, which changes
how it behaves in some way, right? And that changes the distribution that
the charges will follow, right? And when you want
to take something out of the circuit, what you're doing is
observing it. So the circuit will
have a bunch of degrees of freedom that are kind of moving randomly under
the influence of thermal noise. And basically what I can do is I
can hook an amplifier up to the circuit and measure one of
those signals. And so doing that lets me observe
the random dynamics of the circuit. And if I do that over
and over again, as long as I leave a long enough time in between observations,
In the bouncy ball analogy, right, you have your landscape, you poured a
bunch of bouncy balls, still shaking a bit in this landscape, right?
Eventually it would equilibrate to some sort of distribution. Sampling
is like applying a sort of porous grating on
top and letting a bouncy ball sort of hop out. And
from that, you can infer where that bouncy ball comes from. That gives
you one sample, one bouncy ball from the probability mass
of where they all are, right? So that gives you one snapshot. If
you take many snapshots, there's all sorts of algorithms that you
can use those sort of what are called estimators of where the
distribution is as a sort of
signal, either for learning or inferring what values
you're predicting, right? That position could be like the
value of a pixel, an image, right? But
you have probability distributions over everything, right?
Yeah. Interesting. So is that, so the
translation from thermal land back to normal digital
land, I assume still has to happen. Like you still, in order to show something
on my screen, which is a digital screen, like I need to get those
values out. But that's what you're talking about right there. You're saying that whatever we're
basically, you're able to sample and pull out these electrons
or whatever they are, see where they fell, and then that
gives you the value that you need for like a color or for a letter
You're going from this cloud of bouncy balls to, okay, this one is
definite, now I have it, it came from here, right? So, that's a deterministic sample
out of a probabilistic sort of distribution, right? And
so, and there's this old, Yeah,
there's this old thought experiment called Maxwell's
Demon that if you observe a
probabilistic system, it's going to cost you energy to get that information. So for
us, our goal, instead of having to sample
a lot from the device and always have to relay things with
classical computers, we're trying to do as much as
possible natively in probabilistic physics because that's much lower power.
It's going to cost less energy because observing things costs energy.
And that's sort of like what, so one of the things
I understand is like wrong or whatever with quantum computers is that step basically.
Like how do you get the thing out of the quantum, like qubit representation and
put it into a normal, like classical bit. And so are
there similar problems that you run into in this thermal world versus in
the quantum world with that, like basically with the, you
know, that pulling out of the other regime into the normal regime? Like
I imagine in this world to be more specific that you're,
the thing reading the voltage or whatever could be itself noisy. And so you
don't know whether you're actually getting the value that you intend to
get in the translation step out of the thermal system.
So that's where the real work comes
in, right? Is how do I design these various circuits to
In quantum computing, it's called the readout problem, right? It's like I'm
at the quantum regime where we're down to literally few
quantums of energy, right? That's where the word quantum comes
from. We're a few more energy
packets than quantum. We're a bit higher energy than that. But
still, for us, it becomes a problem of amplifying that signal, right?
So ideally, you don't want to have to
have your observations get off of your thermodynamic computer or your quantum computer
into a classical computer and then back. That's very slow. In fact, that's been
a problem with most quantum computers today. If
you try to use them for these sort of quantum deep learning
algorithms, that iteration loop to optimize your parameters way
too slow. Getting those samples out and then getting
that feedback loop update way too slow. And so our
insight is that eventually we want to do that as physics, as
a physical process in the device. And so,
And basically, whenever you want a signal to
travel far, you need to amplify it a lot, right?
Because there's more, like when a single has to physically travel
further through some like weird environment, right? More noise hits it.
And so it's kind of interesting about our approach is you could imagine putting
a lot of this stuff in the same package, right? It's
a CMOS all the way down. So potentially we'll
Huh. Wow. Okay. This is sort of breaking
my brain, but in like a good way. Like I, it's like, it's coming together.
Like, I think, I think I'm picking up what you guys are putting down. Um, I,
so I have a question about like, basically if there's an analog
to like coherence here, like quantum coherence is obviously a
big problem where you can, you did the thing just collapses and
that it like loses its quantum properties. Basically. Do you, does
that happen to you guys when things are, um, like too
big basically because there was a part of the light paper where you said we
got to keep it small we got to keep it low power because then these crazy
So we explicitly don't have quantum coherence, to
be clear. Quantum effects are actually important in
transistors. That's one of the things that limits how small you can make
them is quantum tunneling. But there's a difference between observing
a quantum effect and having a coherent quantum state. Quantum
tunneling in CMOS is not coherent because it's at room temperature. So
that's one tangent. I think the closest analogy
we have to that is if you have a device that's too
big in the right sense of big, you end up
with technically still probabilistic, but
I would say metastable systems in a sense that
if I have two wells with a giant energy barrier between them,
It's very unlikely that thermal noise is going to ever kick you over that barrier, right?
So that thing is going to look more like a digital bit. And so you have to
So for us, it's like the opposite, right? So quantum coherence is
like, it's like the time until your quantum
computer starts to thermalize, until the noise starts seeping in and
affecting things. We want the noise to start seeping in as fast
as possible and for things to equilibrate as fast as possible. So we
have something called the thermalization time. And we want that to be fast.
So for us, it's actually the opposite. We want more noise and
it helps us go faster in many ways. And so
that's the lesson we learned. Instead of trying to extend coherence
times, it's like, hey, nature wants to thermalize. Let
it rip, right? And so let's use that. Let's
use that natural tendency as a building block for our algorithms.
And so it's kind of dual. It's like the opposite of coherence time.
It's like decoherence time. How fast can you decoherence? Yeah,
exactly. So it's like switching sides, you know, half times.
Yeah. Very cool. So what I imagine that when
you both started out on this, and maybe this is a good time to talk sort of about your backgrounds, but
or like how you guys got to, we talked about a little bit at the beginning, but I'd
be curious, like, I imagine at the very beginning of this project, it
was like a glimmer of an idea. And you were like, okay, probably
it's not gonna work, but like, maybe it will. And that will, how sweet would that
be? But then now, you guys have so much further
down that path of like actually building some chips. Like I saw your
little chip at that party, Gil. And like, it's
so, you're so much further down the idea maze and
like the, you know, you must have more confidence now. So I'm curious to
hear, like how much more confident are you that this is actually going to
We have a lot of confidence that the
local neighborhood of ideas we're exploring here with sort of the intersection of
probabilistic machine learning and stochastic electronics is
the future of computing for AI. We
have a couple hypotheses of what that looks like, and we
got a couple bets in that sort of local neighborhood. We're not even married
to one substrate, as you can see. So even in terms of algorithms, we have
a couple bets there. But we're pretty sure that
something in this neighborhood is the future from first principles. And
we've built that conviction from doing these investigations and
having a larger team to sort of paralyze our learning over the
past year and a half or whatnot since the founding. For
me, this idea was a super slow simmer over
the past eight years. Well, eight years if you include time at
Xtropic. And it was an idea that seemed so crazy that
I thought I had lost my mind or something. And I wanted to sanity check
by working in quantum computing, like, hey, we're going to learn a lot
about how to do physics-based computing and imbue AI
into physical systems with quantum
AI, and those learnings will bring to sort of this
alternative form of computation. And so, you
know, that's been a sort of slow exploration in the idea maze, like
Backburner idea, but then I think I think the point of
going all in and burning the boats, right? Like, turning
down every big tech job, selling everything, moving back home with the parents. Obviously,
I don't love my parents anymore. But, you know, that
was a big move, right? Swallowed a lot of my pride, but then I had a lot
of skin in the game, right? It's like, I either make this work... You
only have one life, right? If you have an idea that you think is your
greatest idea of your life, that you think is gonna have the most impact to helping civilization,
you gotta go all in. And so, at that point, going
all in, was what we needed. And then,
you know, getting Trevor on board was a matter of time. Just
had to convince him to drop out from MIT. That
took a couple months. But, you know, and at that point, once he came
in, you know, things really accelerated. Because, you know, we've worked together before.
We shipped TensorFlow Quantum. Again, that was a similar
scenario. It's like, there is no adults in
the room. There's no guidance. The field didn't exist. Big
tech people were asking us where it's going and to
build it. And the best way to predict the future is to invent it
and build it. And that's what we did back then. And that's what we're
doing now again. So Trevor, do
you want to add to that? I'll look for my chip. I think it's somewhere
No, I think you pretty much covered that. I mean, personally, you
know, I feel that computing has to go this way. I've,
you know, I've been thinking about noise and computing and
how they might help each other, how they harm each other for basically like my
entire, you know, academic and adult life. So it's a topic
that's very near and dear to me. And when you kind of combine the
theoretical angles with the fact that we want
devices to be small, when things are small, thermal fluctuations
are important, and therefore devices become noisy, it
all seems kind of inevitable to me. And I
really think what we're doing is inevitable. So in
that sense, I have a lot of confidence. that
Totally. So I think that you mentioned
the category or whatever, like this category of ideas seems right. Ooh, there
we go. There's some hardware. A little metal
There you go. Old chip. Yeah. There you go. It's almost as
big as your people. That's wild. So this one's made out of
aluminum. That's like the easiest process to start with.
There's other fancy superconducting metals we can experiment with that can operate at
a higher temperature. But for us, we
came from quantum computing. This was kind of our lingua franca. And
we know how to build modular physics-based devices out of the substrate. So
it was somewhere to start. But, you know, obviously we have a long
road ahead because, you know, if you're an alien and you don't
have earthly supply chains that are already established, you
build your thermodynamic computer out of this, right? But of course, for
us to grow on a fast timescale, we got to meet in
the middle where existing supply chains add and
find sort of mineral ground. That's where we're going to silicon. I
like to joke that this is the floor of
computers aliens would build from first principles. But
of course, if you have the deep pockets to
scale up superconducting technology, you might be interested in this and you can give
us a call and we can work with you. Including
the aliens, they're listening to this. Give
us a call. But hey man, I don't judge.
So yeah, essentially, yeah.
So you mentioned the broad category that, you know, this idea is
a part of seems like the right one. It's, you know, for physical reasons,
for like, you know, like the frustrations you had in quantum reasons. are
you guys are one of one? Are you guys the only people thinking about
this? Or I know that like neuromorphic computing
is broadly a category, but I don't know if it's like
somewhat applicable to what you guys are doing. Do you consider yourself part
of that broader subfield or more
I think people have been obsessed with sort of biomimicry, right?
They're like, well, if we obsess over every details of
how neurons actually work and we mimic that, something's going
to work, right? We're going to figure out how to program it later. Whereas for us,
it's like, no, no, no, that's not how it works. You got to like start with the algorithm and
then, you know, both top-down and bottom-up sort of engineer this
bridge between what you want to do and the physics of the
device. And that's what we're doing. We've established this sort of full stack bridge. And
that's so interdisciplinary. It's so difficult because you need to have like ML
people talking to physicists, talking to compiler people, talking to hardware
people. It's a very difficult effort, but
we did it before in
Give me a second here. A comment there. Computing
has kind of started as this abstract thing where a
computer meant like a Boolean logic machine, right? But
in the 21st century we're actually starting to see things go
a different way in a sense that computing is
just becoming kind of more widely understood as just
embedding math into a physical process, right? This
started to become really obvious in quantum computing because the
way that people have been successful in quantum computing to date is
you start with the physics of your device and you see what
kind of computations it does relatively naturally, right? Those are
going to be the things that are going to be highest performing. And back
at Google, like the things that we've gotten working on quantum
computers to date are all kind of
things that very naturally map to the physics of the qubits, right?
Padram Rishan, Vadim Semyansky, that's the kind of game they
play there and it's been very successful. So I'm kind
of taking that approach to computing and
bringing it to like room temperature devices that scale, right?
And ultimately what that's going to do is it's going to kind of hack the
last, every last drop of performance out
All right, yeah, what else do you guys want to talk about?
Yeah, no, please. I had an analogy. So an analogy we like
to use about sort of biomimicry versus what we're doing, right?
If you set out to build a flying machine, right, you're
like, oh, well, you know, the proof of existence is out
there, we have birds. right? Birds flap
their wings. They use some form of physical principle we don't quite understand. Let's
make a plane that flaps its wings, right? And
that's going to be the device I make to achieve flight. On
the other hand, you can sort of go up the supply chain
of nature itself, right, of biology. Ultimately, biology
just finds a way to hack some sort of principle in
physics to its own advantage. And so, in this case, it was like
the physics of lift, right, or flight. And
so, when you build an airplane, an artificial system, you
just try to build the best system that
leverages that physical principle that biology found a way to leverage,
not obsessed with biomimicry. So, neuromorphic devices are obsessed with
biomimicry. We just understand how
natural systems leveraged out-of-equilibrium thermodynamics
in order to do probabilistic machine learning natively
as a physical process, and we're building devices that are better
than nature. Like, our neurons are, the superconducting chip, far more efficient than
your brain. Right? Which is astounding because your
brain is like tens of millions of times more
energy efficient than GPU clouds today and much denser. And
we're going more denser and more energy efficient than the brain, which
may scare some people. But, you know, to
us, in order to be able to understand
and predict our world at all scales, There's just so much
intelligence needed for us to scale civilization that we
just need to accelerate as fast as possible and reach the end We're trying
to reach the end of computing, right? We're trying to reach the ultimate substrate
for intelligence in terms of energy and efficiency and density because
that's where everyone is going And so we're like, let's just
go there right away from first principles and see how
far we can get. And I think we can get pretty far. And so that's
what we're going after. So we're taking inspiration from
nature, but really we're just trying to hack physics directly. The
What's interesting about this neuromorphic space, or
neuromorphic or physics-inspired computing, whatever you want to call it,
is that every different kind of device has
its own kind of natural set of algorithms that
it can accelerate, right? Because building a physics-based accelerator means
you're embedding some kind of math that you want the answer to
in the dynamics of an analog system. Right? And
so when you build dramatically different devices, maybe
like a quantum computer is really good at solving the Schrodinger equation, like
a memristor array type of thing is really good
at simulating memristor arrays. Our
computer is really good at sampling from programmable distribution. So
in a sense, the point I'm trying to get at here is there's room for
a lot of different plays in this space because every
accelerator ends up being good at something different. So
in that sense, I don't think there's any real competition out
What do you think the main applications are going to be just that fit
today's world? So, you know, I imagine that you guys will invent some new
stuff, some new software algorithms, your own software, but
is there going to be a one-to-one analog for people that are doing normal
models today that they'll be able to, oh, they'll just plug your thing in instead
Yeah, we definitely want to support current day, you
know, deep learning and machine learning practitioners. Of course, for
us, those applications like large language models are
applications we achieve at scale, right? When our devices scale, because there's
large in the name of large language models, right? And so there's
a lot of machine learning models that are more valuable
to businesses in some ways. that are in the low data regime,
where you need to have probabilistic uncertainty about your predictions, right? Let's
say you're, you're doing a trade, you're pricing options, or, you
know, you're trying to optimize the manufacturing process, every
data point can cost millions, if not billions of dollars. you
don't have that many data points, and it doesn't matter how much compute you want to
throw at it. You want the best possible answer, and
you want to quantify how uncertain you are about your predictions. That's
the sort of algorithm we're trying to enable in the short term.
And so, did I cut out there? No,
I did not. Okay. I did that. So that's the sort of algorithm
we want to enable. And so that's a different regime
than the big data, big compute regime or big classical compute regime. We're
liking the extreme compute regime. We harness a lot of
that compute from nature, right? The probabilistic compute for practically
free just from heat from nature. But we're
going to tackle sort of low data regime probabilistic
algorithms, right? And we think that these are
actually in some ways more useful than
large language models for businesses. Or
at least a nice sort of dual to them or another
type of machine learning that is synergistic. And
so for us, sort of the LLM market
is interesting. That's where we want to get to. But in the early days,
it's going to be these sort of other algorithms that maybe are more popular in
Let's just say that. Like how there's lots of room in
device space for new things, I think there's lots of room in
AI space for new things, right? It's easy to get caught up in
what's most important today, but you have to take a second and
have some perspective. We're really early in
AI. This is like the birthing years, right?
And so the technology that, I mean, there's never going to be an
ultimate technology. There's always going to be a new and better thing. But, you know, looking
10 years down the line, 20 years down the line, that stuff
might have no resemblance to what we're doing today, right?
This is the next S-curve. We think that current AI,
it's scaling, it's very impressive. It's going to hit some
sort of saturation, might be the data bottleneck. I think definitely the
energy bottleneck, right? You've got
to move mountains. 7 trillion. 7 trillion, right? Just throw 7 trillion
at it. It's going to fix everything. you know, we think there's a
better way. We're already working on the next S-curve, right, after
this one. So it takes time to ramp up to the level where
we're at the state of the art, but we know that, again,
by 2030 or so, we're probably gonna hit a wall in
terms of scaling down our current deterministic transistor technology
because they're gonna hit this sort of thermodynamic regime. Their wobbliness
is gonna be a problem. We're building beyond Moore's wall. So we're
gonna enable us to extend Moore's law in a sense,
just for AI and probabilistic computing, not
for general computing, but that's still great because that's where
we want the extreme scale compute anyways. And
so to us, this is the most important thing we could be working on in our lives. And
every day we just wake up with insane levels of internal fire.
We're on a mission to save the world here, and
we kind of hyperstition this, and we're kind of in a position where it
is the case, and it's kind of surreal, of
course, and there's many other things going on in
our lives as well. Most
people's priors is that if we had a successful movement,
couldn't be successful in technology, but for me, I think this
technology, you know, the cultural movement is great and I want more people to
join in on the optimism and to have life paths similar to
ours, if possible, because then everybody would accelerate. But,
you know, ultimately, for me, this mission is
the one I'm like most passionate about. And, you know, I'm
all it, right? This is the meaning and purpose of
our lives. I truly believe that. And that
just gives a deep sense of satisfaction working on this stuff
every day and gives you near infinite
energy somehow. It just comes out of nowhere, right? Like if you have this infinite
goal and you're making progress towards it, it
feels really good. And so any other bump in the road, any challenge, it's
just a temporary setback on this road to
an amazing goal. And so couldn't be more excited to finally, talk
more about it today. It feels very cathartic right now to talk
about it in a sort of public podcast. These have been
like our internal secrets for a while on our
thesis, but the reason we're showing more about
the world is so that people know. People know it's coming. And
if you want to work on this sort of stuff, if you're a talented builder, if
you're ready to run through walls to do this, you should give us a call
So we got four new job postings, but really it's
whoever wants to join us on this journey and believes
in the mission now that we've kind of laid it out, you
know, should talk to us. And so, you know, our goal
is for everyone to accelerate and, you know, in the ethos of, you
know, some of our techno-optimistic thinking, we're putting our
ideas out there. And hopefully
the universe will reward us back for creating
all this value, but we're going to keep going no matter what, because it's the most
And by talk to us, he means apply to our job postings, because if you
DM us on Twitter, it's very likely not
Yeah, I get a lot of DMs. Yeah, yeah, yeah,
yeah. So ideally job posting. Yeah, so
that's our goal today. I love it. Thanks so much, Christian. I
Oh yeah, absolutely. I mean, you guys are the perfect people to have on it. I feel like you
name-dropped first principles many times throughout the podcast, which I am very, very
happy to hear. Awesome. And I hope that people, so people
should know if they didn't know this already, and it'll be linked in the show notes, but there
is a paper we were talking about through this, so they've actually published some
It's just a few ideas to get you thinking. It's
Not yet. That's right. Well, they did have that conference in San Francisco yesterday
that was like how to build a nuclear bomb. So who knows, maybe they're- Right. No,
thank you guys so much for joining. This was awesome. And who knows, maybe
when you get your next chips or when you publish a full white paper, we'll have another one of these and go
That sounds good. Sounds good. Thanks so much. Awesome. Thanks, Christian. Thanks, guys. All right. Cheers.