Robin Hanson: Prediction Markets, the Future of Civilization, and Polymathy — #66

Robin Hanson is a professor of economics at George Mason University. He has worked in a variety of fields, including Physics, AI, Economics, and Futurism.

Follow him at https://x.com/robinhanson

"When the typical economist tells me about his latest research, my standard reaction is 'Eh, maybe.' Then I forget about it. When Robin Hanson tells me about his latest research, my standard reaction is 'No way! Impossible!' Then I think about it for years." -- Prof. Bryan Caplan, GMU

0:00 Introduction

00:34 Welcome and Manifest conference introduction

03:12 Robin Hanson: Education and Early Influences

08:38 Transition from Physics+AI to Social Science and Economics

22:02 Prediction Markets: Potential and Challenges

28:37 Cultural Drift and Challenges to Modern Society

40:49 Fertility and Demography

48:37 Life as a Polymath

59:27 Future of Artificial Intelligence and the Simulation Question

01:09:29 Audience Q&A

Music used with permission from Blade Runner Blues Livestream improvisation by State Azure.

--

Steve Hsu is Professor of Theoretical Physics and of Computational Mathematics, Science, and Engineering at Michigan State University. Previously, he was Senior Vice President for Research and Innovation at MSU and Director of the Institute of Theoretical Science at the University of Oregon. Hsu is a startup founder (SuperFocus.ai, SafeWeb, Genomic Prediction, Othram) and advisor to venture capital and other investment firms. He was educated at Caltech and Berkeley, was a Harvard Junior Fellow, and has held faculty positions at Yale, the University of Oregon, and MSU. Please send any questions or suggestions to [e-mail] or Steve on X @hsu_steve.

Creators & Guests

Host

Stephen Hsu

Steve Hsu is Professor of Theoretical Physics and of Computational Mathematics, Science, and Engineering at Michigan State University.

What is Manifold?

Steve Hsu is Professor of Theoretical Physics and Computational Mathematics, Science, and Engineering at Michigan State University. Join him for wide-ranging conversations with leading writers, scientists, technologists, academics, entrepreneurs, investors, and more.

Robin Hanson: All innovation is a few elegant ideas and lots of messy details, and it doesn't end up working unless somebody works out the messy details. Yeah. Okay. So this is an area where we haven't worked out those messy details because we haven't had practice. but every plausible innovation is also a social science opportunity to update your theories about the world.

You have a theory that tells you that this should be good. And then you go look at the world and their demand is much less than you thought. And so this is an opportunity to learn.

Welcome to Manifold. My guest today is Robin Hanson, a famed economist and freethinker and polymath. We are here at the Manifest conference sponsored by Manifold markets. It's a crazy rationalist EA fun fest here at the Lighthaven campus in Berkeley, California.

Steve Hsu: I highly recommend it. This conference to listeners of this podcast. Robin, welcome to the show.

Robin Hanson: Thank you. And it's especially prediction market themed. Yes. This conference, not just Rationalists.

Steve Hsu: Yes. And so among the many, many things I could discuss with you, we're gonna at least spend a little bit of time on prediction markets.

A, because you made a major contribution to that field, but also because it's a theme of this meeting that we're at. So for people in the audience, we are taking questions. our associate who is off camera. Is monitoring my Twitter feed. So if you look me up on Twitter, Steve shoe, and you reply to my post, asking for questions to Robin, we will select from the best questions and introduce them into the conversation.

And that goes for people who are here in the audience, as well as people all over the world on Twitter. So Robin, we met, I think a little over 20 years ago. you and I are both at a conference at. HP labs. I had been invited by a guy who was a researcher at HP labs named Bernardo Huberman. That might've been the guy who invited you as well.

It was a great pleasure to meet you. I think your talk was actually on the mathematics of prediction markets, which I think was one of your major focuses of work around that time. Yes. And then we also had a long discussion of, I think, the Allman agreement theorem. Also working on. Yes. Okay. So, it was a great opportunity to meet you.

And, so I've always thought of you as one of the best examples of a polymath because you're capable of making contributions in so many different areas, you have kind of unlimited interests, and you're also not afraid of doing things, which are marching to the beat of a different drummer. So what I wanted to do, at least in the first part of our conversation is.

Talk about a little bit of your history, because I'm interested in intellectual history. So how does a great mind evolve over time? What are the main influences? What were the things you hoped to achieve when you were younger? How did you view the world when you were younger? So let's just start with your sort of high school, college days.

What were your interests then? And what did you think you were going to do in life?

Robin Hanson: In high school, I took a physics class. And

Steve Hsu: as one does,

Robin Hanson: I was charmed by it. Because I was not asked to believe things and spit them back that teachers supposedly proved them to me. That was revolutionary to me. Because I, I guess, was inclined to be skeptical.

so being willing to be skeptical will be the sort of thing that lets you march to a different drummer and do different things. But you'll have to at some point be willing to maybe not trust what you've been told. And there it was. They were showing me what they were claiming. And so am I. And that's probably what pushed me into physics later on was that, again, I didn't just have to believe them.

I could check, I could, you know, look for internal consistency, et cetera. That's in a sense, the charming thing about physics to a young person is they give you a relatively small number of concepts, a relatively small number of claims that you can connect better, check on each other, and then check against lab results and see that, okay, it works.

I can believe that. And you don't have to trust people.

Steve Hsu: And Were there particular ideas you found beautiful or attractive, in the physics curriculum?

Robin Hanson: II was focused on sort of the underlying coherence and, you know, whether there were holes or left out parts or open questions or mysteries, I was less focused on practically using it than seeing it as an overall coherent hole and what can it make sense of.

So actually physics was organized at my university in two year chunks. So, you went over six subjects in three quarters each year in the first two years, and then you went over exactly the same six subjects in the last two years with more math. And I looked at that when I was at the two year mark and I said, more math, but I have questions.

Like a lot of these things didn't fully make sense. I needed to understand them. So I took a very unusual strategy. And when I look back on it, I'm somewhat amazed that I was just arrogant enough to do so, but I just decided. Not to do the homework, because that was very time consuming, and just play with the equations all semester long, and ace the exams.

And, you know, that's, I guess, evidence from the outside and from me looking back that I just was willing to go my own way and ignore the curriculum to some degree because I wanted to figure things out.

Steve Hsu: And as your senior year approached, were you thinking you would continue in physics or, and, and at the same time while you were learning physics, were there other subjects?

For example, you're now teaching kids economics, I think. Right. Have you taken any economics courses at UC Irvine? The

Robin Hanson: The only social science class I took was by a political scientist called Taagepera, and his class was almost entirely games. He taught political science in terms of games. So we basically played games, which had various lessons about war and things like that.

And That was actually engrossing and interesting, but it never really grabbed my energy, although later on I became a social scientist, right? So it's interesting to think, well, there was social science in front of me through game theory, through concrete games, which I love. And it didn't engage me.

So there's, I guess there's this old story, which is somewhat true that, you become more interested in social science as you get older, as you see the social world around you and you start to realize how much it matters and how puzzling it is. And then you aren't ready for thinking about social science and you're just not ready for social science when you're just figuring out how to think.

Yes, that was me. Yes. So did you, did you do a master's in physics also? Or did you, I did a, I did a master's in physics, but I, I initially decided I was going to get a PhD in conceptual foundations of science or philosophy of science. Because again, I had these questions about what the hell was science. So all through physics, I was questioning all these sorts of constructs of physics and whether it made sense.

But then there was this process we were going through and the process supposedly created the stuff I was learning. And there were these stories they would tell me about where this stuff came from and how it made different. And that just didn't make sense to me. And I wanted to know, okay, well, what is this science stuff?

Apparently it's crucial for making all this stuff I've been learning. And so I heard that philosophy of science is where you learn that stuff and science studies is where you learn it. So I decided to go get a PhD in science studies at University of Chicago, which was, I guess, one of the top schools then at the

Audience member: time.

Robin Hanson: And Again, again, I was just amazingly looking back. I just had questions and I was just going to pursue those questions. I had no sense of career or prestige or impressing people or joining a team or making a team or having, I just had questions that I was going to try to figure them out. So for philosophy of science, I think after a year or twoI think I kind of answered the questions that came in with, That may be less interesting.

So I went back to physics. So to get a PhD there in conceptual foundations of science, you had to get a master's in the field of science. So I had started on that in physics and then having answered the questions, I moved into physics back and then I still read more things and I had more questions. And so after three years, I went off to Silicon Valley because I had read about artificial intelligence and I had read about hypertext publishing and those most exciting and potential for doing things.

And that pulled me away from physics. And this was, what year was this? 1984 is when I left the University of Chicago to go to Silicon Valley. But, and so 1981 is when I arrived in Chicago to study philosophy of science. And I guess, again, I just had these topics and interests and I wanted to go pursue them.

But I think artificial intelligence and hypertext publishing, in contrast with the others, had this vision of, Of something being created

Steve Hsu: and

Robin Hanson: new things were going to be created, not just discovered or learned. And I wanted to be part of that creation. I wanted to make AI. I wanted to make a hypertext and that inspired me.

Steve Hsu: And looking back, were you doing AI during what we now call the AI winter? Is that right? I

Robin Hanson: started AI in 84. So that was still a high activity period of AI. Okay. And part of what drew me in is with all these newspaper articles, et cetera, talking about how AI was going gangbusters and it would all be over soon if you didn't get in there soon.

Steve Hsu: Yes. We're close to the singularity.

Robin Hanson: Yeah. Yeah. So I learned my lesson, I guess. So ever since then, I'm a little less susceptible to them, we're almost there and it's going to be a big boom. You need to get in now before it's too late. As I, as I fell for that once.

Steve Hsu: Yes. I'm going to return to that topic later in our conversation.

So, somehow you decided though, to go back to graduate school, and you ended up doing a PhD in economics at Caltech. Is that right?

Robin Hanson: So I, in 84, I went to Silicon Valley and I had these two topics I was interested in because I both heard they were there. AI, I really like Doug stuff. And he was at Stanford and for hypertext, this company Xanadu.

People were there and they were about to do things. So that was the place to be. So I went to Silicon Valley, but of course I didn't have any skills in these areas. being the arrogant guy who just follows his questions. So I went to get a job at Lockheed as an engineer, but within a year, we weasel my way into the AI center because I guess I had, you know, a good background or whatever.

And then on the side I worked with the Xanadu Hypertext people, and I was pursuing both of those things. So I learned a lot about AI. And to a physicist, I think most physicists of the sort that I was don't really have a concept of just how much other things there are out there that aren't physics that you could learn.

So it was a surprise to me to discover all these things to learn in computer science and AI that I hadn't known and that was just fun. and then on the side with hypertext, the Xanadu became a firm and they started pursuing things. And, well, I kept thinking about these things and Xanadu's vision was initially compelling to me, and then I came to question it.

so their vision was backlinked. The idea is when you go read something, it could be wrong. And in the pre hypertext world, You wouldn't know to question it because you couldn't find the rebuttals. But in a world of hypertext, if somebody had linked to that, you could follow the link backwards and see all the criticism and the ideas that would make our world much healthier.

We would no longer believe stupid stuff because we could find the rebuttals. It hasn't quite worked out that way. As you know, after a few years, I started to question that. I thought, is that really going to work that way? And these people around me were rapid libertarians. For some random reason, libertarians were into making these new technology things that were a correlation back then.

And that put this idea in my mind of betting markets as a thing you could do and might be okay. And then I thought, well, couldn't betting markets do this fix that you could, that we were trying to do with hypertext, wouldn't that like head out the bullshit and get reliable answers that we could do if only we would bet on these important topics.

And so I started to think about it. Betting markets and what you could do with them and started to talk and, you know, try to advocate that and talk to people about it. And that pulled me into more social science and to thinking about how the social world works. And after a while I decided that being a nobody with no contacts or credentials wasn't going to go anywhere.

And I needed credentials. And if I decided I was going to do something, I had to pick something to do. And I actually decided, oh, I should do science studies, go back to that, and apply to a program. They had accepted me, and I had accepted their acceptance, and then a few days later I said no.

Changed my mind. Spent another year, and then I said, I should do economics, or social science, because I had read about people doing experimental economics and as a physics person, that was compelling to me. And you know, like a physics person, I had never believed in economics or social science. We were all told that's all bullshit, but we were told experiments are gold.

So I thought we need to invent new institutions or you could just do experiments to test them. That's what you could do. So I decided I'm going to go off and get a PhD. In

Steve Hsu: social science. Quick question. Is it true that you actually instantiated a prediction market at Xanadu? And that was, was that the first one ever for this kind of

Robin Hanson: thing or these corporate internal corporate predictions?

It was a market. It was internal to Xanadu. It had some corporate relevant topics. It's the first one I would know of those things of that form. And we did use my automated market maker, a mechanical version in the room there. Okay. of course, betting markets are ancient, so I can't claim to have had the first betting market.

Yes, but I can claim to have had the first internal market designed to address internal questions. So interestingly, Xanadu project's main failing was they just kept having higher requirements because they wanted to solve everything before they released this thing, and they never did. The World Wide Web by Tim Berners Lee was introduced and spread While they were still trying to get their code released.

So they never ended up releasing it. but they had a market at one time about whether they would release their market before the then current premier of China died. So they thought, well, our thing would be helpful for those people in China. So we'd like to release this in time for that crisis. But of course they never did.

So what year did you start at Caltech? Caltech? That would

Steve Hsu: being 93. Okay. Okay. So I was already gone by then. We didn't, we didn't overlap on campus. Now, when I was an undergrad, I participated in a bunch of experiments that were run by, I think these economists, there's a guy who later won the Nobel prize.

Smith? Yes. Vernon Smith. Vernon Smith.

Robin Hanson: And,

Steve Hsu: um. He wasn't

Robin Hanson: at Caltech, but another person who was a common worker in the field, Charlie Plott, was at Caltech. Okay.

Steve Hsu: Okay. And, did you end up, did you end up running those

Robin Hanson: experiments when you were a grad

Steve Hsu: student there?

Robin Hanson: I participated in experiments. So. I went to Caltech initially because I thought this is the way to test the ideas I'm interested in, not just prediction markets.

I had a couple of other ideas. And when I got there, I learned there was a rule that I hadn't anticipated, which was we don't test mechanisms. You want to come here and test mechanisms. We don't test mechanisms, but it looks like you're testing mechanisms. No, we only test game theory models of mechanisms.

So we'll run the mechanism. If you have a model of it, then we can compare. The data on the mechanism to the model of the mechanism. I just want to know if it works. We don't want to know if it works. We want to know if the game theory model of it is accurate. Okay. Well, that was a big obstacle because making models of these mechanisms a lot harder.

And so that distracted me into making models of things, learning game theory. Wellbut in the process I realized. Just as I did in computer science, there was a lot of stuff I didn't know and a lot of these insights were powerful and so I became basically more of a mathematical economist who, you know, did formal theory of various applied economists, I guess, which is what most Caltech people were.

So, right. I mean, I participated in experiments and I helped design some but I wasn't, that wasn't the main route I went. At the time, the FCC was doing spectrum auctions and they were advising the spectrum auctions and they specialized in combinatorial auctions. And I'm, I probably came up with some clever combinatorial situations where you'd need a combinatorial auction to do it well because otherwise the regular auctions would fail.

Steve Hsu: So Vernon Smithlike you originally started by studying physics and then eventually became an economist. And, in interviews that he's given, I find him quite an interesting figure because he is very open about being autistic. And he actually says that he's able to come up with ideas that, Seem straightforward to him, but in his profession, people are shocked by them because he doesn't register the sort of social feedback, which would make you kind of conformist or, or whatever it is.

Do you see, I don't know if you're familiar with that aspect of Vernon Smith. I don't know about him, but that's something I've said about myself independently.

Robin Hanson: I don't think I got it from him. Okay. I think it's just generally true that most of us glide through the social world, functionally through intuition.

And then we have these explicit Models in our head about what people are doing that are not, are in contradictions to our behavior, but we don't notice that. We just have the, you know, everybody's nice to each other. People could trust, you know, all these sort of nice things we like to say. And then we actually are pretty functional in how we interact with each other, but we just don't see that contradiction.

So if you're pretty nerdy, artistic, then you don't glide so smoothly through the social world. You bump and get a lot of bruises really. And so. You will then maybe notice the contradictions between the things people are saying and the things you might be inclined to say and what's actually happening because you need to more consciously navigate the world around you.

And that means you are noticing that the things you're seeing are not what people are saying. And so that's an important prod to help me to be an economist. If your audience is willing to believe that in fact, the usual stories are wrong, which many are not. Yeah. I think the physicist's part makes you just Arrogant and ambitious and expect simplicity.

Yes. that is, and that, that, that has carried me on. So that's likeVernon and myself, for many people who didn't come from physics, they just expect this vast irreducible complexity from which they don't expect to find much simple patterns and they don't expect to be able to predict very well, and then they don't have the ambition to look for a big, simple pattern that would make sense of things.

but physicists definitely expect there's a structure and it makes sense of everything and it all fits together. If you haven't found it yet, you should keep looking for it because it's there somewhere. And I think that definitely has made me more ambitious. They also just make me want to integrate my view of all sorts of fields.

It's a polymath. So just having one field and going to another is going to push you more into wanting an integrated view of everything, which most economists don't have. Yes.

Steve Hsu: The same thing has been said for, there are many physicists who went into biology and made some contributions and the arrogance and the expectation that there is some, could be some simplistic structure, simple structure that you can elucidate, even in a really complex system, like a living organism, that, that turned out to be an advantage in some areas for, for physicists.

Robin Hanson: But of course we're often wrong. We ex physicists are often wrong. And yeah, this is like in economics, there's like econo physics. Which is the name for people like just taking physics models and chasing the labels and calling economics. And it doesn't tend to work very well. So economists are often pretty arrogant about their attempts to project economics elsewhere, but still, I mean, one basic fact about research is in general, the downside is pretty low.

The upside is really high. So high variance is often pretty good. Like. Okay. So, these econophysicists don't do so well. They don't hurt much either. Yep. And the few times when you go into a field and you revolutionize it, well, that's well worth all these times you don't do much.

Steve Hsu: Yep. So let's, let's jump ahead now and talk about prediction markets since we're here at Manifest.

can you give, so my audience is probably not that familiar with prediction markets in general. So could you give a very kind of brief introduction to, how you got into it and, and, and where you think the field is now.

Robin Hanson: So for centuries, people have bet on things and the usual reason betting markets exist is because the people who bet want to do so either for action, which is sort of a gambling impulse.

You just want to take a risk and maybe win. Or for reputation or pride, you want to show people that, you know, like I met, think of a bar bet one, sir. My assertion says, Hey, no, I dammit, not a, and I'll bet, I'll bet you, right. It's a way of showing that you think you're confident in something and, Establishing a generous attitude.

So that's been going on for a long time. The new thing here is to ask, what if we wanted a better answer to a question? Would we be willing to pay to predict a betting market basically in order to get the answer? So. The betting market that somebody pays for looks the same. So if you imagine you actually wanted to know which horse would win a horse race, you could, if people weren't willing to race horses without your extra input, you could actually pay to have a horse race and see who wins.

And if you wanted to know who would win the race before the race happens, you might introduce a betting market where people will bet on the horses, and then you could use the odds. As an indication would win now, you might not reflect on that for a moment, but basically a betting market is like most any speculative market in the world.

It's pretty much the same structure. And the key idea is there's a price. Everybody can see the price. Everybody can ask themselves, does that price sound right? And anybody who thinks they have any reason to think that price is wrong, has an incentive to trade in that market and their trade will move the price in the direction they think is right.

When thousands of people do this over and over again, the price ends up embodying a lot of information. It represents a lot of knowledge about that. That's just how this mechanically works. And in fact, if you look at all the other ways we have to take a group of people, put them together and have them figure out some common estimate about what they all believe, this beats them all, at least in many contexts.

Many contexts we've tested, this is at least as well or better than other things. So I won't say it's always the best way to collect information, but It's damn impressive. And we have all these contexts where it looks like we disagree and we can't figure out how to put our information together. And it looks like there's a lot of bullshit.

And then this looks like a powerful solution. All you have to do is subsidize this enough to get people to bed. Look at the odds. And there's your answer. I mean, that's just this crazy potential. I mean, you know, if you think about all the things we don't know and all the things we want to know, most of them can be put in a format much like this.

So why don't we do this

Steve Hsu: now? I think after I won, I think I was exposed to your work on this at that first conference when we met. And it was, I think many years later, I think I'd seen you in other places too, but we, we bumped into each other again at right at the beginning of the crypto boom, it was a conference, I think on Ethereum and we were sitting together at lunch and then I sort of asked you, so what's, cause I read in the papers or in the media over time that I think you had gotten some companies to actually instantiate or maybe the government.

The defense industry to instantiate some prediction markets. And then when I sat down with you at lunch and we were talking about it, you seemed kind of pessimistic. Like, I think you more or less just, this is like a stylized memory of what you said. But, I think you said something like, well, the markets work fine, but the executives don't want the information out there.

They want to control their companies. And so eventually they end up killing the markets. Right. And, and so, It sounds like my memory is correct. And now where are we? So, so has that situation changed? manifest of course, or manifold is the marketplace is just available to everyone. So it's in a way almost like a public good, where do you see the future of prediction markets?

Robin Hanson: All innovation is a few elegant ideas and lots of messy details, and it doesn't end up working unless somebody works out the messy details. Yeah. Okay. So this is an area where we haven't worked out those messy details because we haven't had practice. but every plausible innovation is also a social science opportunity to update your theories about the world.

You have a theory that tells you that this should be good. And then you go look at the world and their demand is much less than you thought. And so this is an opportunity to learn. So I have tried to learn a lot over the years about what is this telling us? and I think basically The world can just be in equilibria where they don't want this, but then there are other equilibria where they, if they were doing this, they would not want to get rid of it.

And this is actually true for lots of useful things, lots of useful things. For example, cost accounting, cost accounting is great. It's wonderful. Right. But imagine a world where nobody did cost accounting. You come to your organization and say, how about we do cost accounting here? The message people would hear is somebody's stealing around here.

We should find out who. That would not be a welcome message. Right. But now imagine a world like ours where everybody does cost accounting. You say, Hey, on this project, let's not do cost accounting. What will people hear? Could we all just steal and not keep track of it? Not a good message either. So I think prediction markets are like that in a world where nobody does them.

And you say, Hey, let's do a prediction market on our deadline. People hear that as saying, you think our current estimate of the deadline is bullshit, right?

Steve Hsu: And

Robin Hanson: You want bullshitting. That's not a welcome message to people around, especially if they're bullshitting in a world where everybody has a.

Prediction market on a deadline. And then you said, Hey, let's not do a prediction market on our deadline. People would hear, we're not going to make this deadline. Could we just not look at that? It would also be a bad message. So there are many innovations where basically the world could be better off with them, but in an equilibrium where they're not existing, it's hard to introduce them and in a world where they were common, it would be hard to take them away.

So that's my long term hope for prediction markets is we can get over the hump and into the world where. You would be embarrassed not to do

Steve Hsu: it. Great. So now we're on the topic of prediction markets. I'm going to ask my assistant here. Are there any questions that came in through Twitter on this particular topic that we should hit before we move on to another topic?

Audience member: A couple of minutes ago and nothing on this topic, lots of questions coming in, but they are very topics.

Steve Hsu: Okay. which I'd

Audience member: like to hear the answer to, but I don't want to jump to that.

Steve Hsu: Okay. I want to talk a little bit about demographics and culture, is it cultural decay? Is that? Drift. Drift, cultural drift.

Solet's talk about that and then we'll go to the, the audience suggestions. maybe you could give a short per se of your cultural drift idea. And I'm particularly interested in the impact on fertility and demographics. As you should be. Yeah. If you could focus on that.

Robin Hanson: So I've been a futurist most of my life.

You know, a tech futurist thinking about new institutions and mechanisms like prediction markets and many other things. My first book called the age of M is about brain emulations. What the world will be like. I've for a long time been assuming that growth will continue and technological change will continue at previous rates or even faster.

And that within a century or two, that will go enormously different places. And in some sense, dramatic and even scary places, but. That's energizing. Like let's, let's see, let's go. That's really exciting. Those of us who see that sort of future are very energized by it and wonder why the rest of the world would seem so indifferent to these exciting technological growth futures.

Right? So it's pretty embarrassing. And the downer here to all of a sudden decide, Oh, that's probably not happening. We probably face a several century decline of our civilization where Innovation grinds to a halt and we don't have much in the way of innovation for a long time. And what we stuck at, whatever technical levels we stopped at, and maybe even decay from those.

And actually even worse, not only will our civilization decline and decay and fall, but most likely way that ends is where Heretim, who are doubling every 20 years and half for a century, just replace us and choose whatever civilization they want and This is like how the Romans replaced, I'm sorry, the Christians took over the Roman Empire by doubling every 20 years for three centuries.

And the Romans at the end, when they took over, they threw a lot of Roman things away. Other Roman things had just decayed and gone away naturally because of the flood. This is what our civilization faces, and that's pretty sad, really. I'd like to find a way to prevent it, but so far I haven't explained to you why this is at all plausible.

I didn't think this was at all plausible. And all of a sudden, here, this is what I'm saying. So, the word culture, I'll bet, evokes, you know, symphonies and art museums. And, you know, music, literature, the word culture is almost the opposite side of the aisle from STEM folks. You know, that's the flag that it's those people in charge of that.

And they're talking about it. And we STEM people just like to run away from that and go talk about STEM stuff. And, so I had never thought so much about, and even as a social scientist, whenever people brought up the word culture, it was like this flag for sort of being fuzzy and fluffy and just like, let's not get into any detail.

Let's just sort of point in the vague direction of countries or places. And, you know, try to talk about that. And I'd always seen it's kind of bullshit to me. So I'm embarrassed to realize that, okay, culture's a thing. There's literature about it. I can learn that literature and I can figure things out from things.

And that's the thing that tells me where we have deep, deep problems. So when I came to this for us from fertility, that is a year or so ago, I realized this fact that other people have discovered, not me, that a falling population, innovation grinds to a halt. And innovation is really important in the economic and tech worldview.

So that's a future you go, what? And so I spent a long time trying to understand why and what we could do about it. And I realized that we do know roughly proximate causes for fertility decline. And I can list, you know, six or seven and they are approximately cultural trends. Cultural trends are the proximate causes of fertility decline.

For example, more intensive parenting, you know, capstone as opposed to cornerstone marriages, more gender equality. you know, less grandparent involvement, a whole bunch of trends that are familiar trends, their cultural trends, their beloved trends. And those are the proximate calls that cause fertility to climb.

And then it went, I went, well, okay, well, what, what causes that sort of stuff to happen? Like, why does culture change? And then, digging further into the literature on cultural evolution, I came to understand it. It's not actually that complicated, but a lot of people don't seem to, so a lot of STEM people.

Love says DNA evolution. They're all over that. They love it as a concept. They like to quote it and like to apply it to things for some reason. They don't love and apply cultural evolution. That's they're kind of out of date because for humans, cultural evolution, where it's, it's where it's at much faster, exactly.

And so it's well worth it, if you're, if you're going to bother to understand. DNA evolution, you should bother to understand cultural evolution because it's the thing you should apply to the world around you. And. The shocking thing is that cultural evolution allows cultures to change and no one's in charge and they change fast.

So, let me just walk you through it. The superhuman superpower is the ability to change our behavior much faster than DNA allows by copying each other. The simplest version of copying each other was to copy the high status, copy the successful. And that. That works. So cultural evolution is like a sports car, except it's a prototype and it doesn't have brakes and all sorts of things, but it works much faster than everybody else's tricycle, but it's new and hasn't been worked on.

So DNA evolutions, all these bells and whistles that make it work really efficiently and handle all sorts of cases. Cultural evolution is really crude and fast and breaks. So the first form of cultural evolution, just, you know, high status, copy the high status that can go wrong. If we pick the wrong sort of status markers.

Yeah. but it still goes okay, but then we developed a better version, better versions where we copy specific behaviors, like copy the fishing behavior of the people who catch the most fish, that's much more reliable, doesn't go wrong so easy, and that's the basis of technology, and that's going wonderful in gangbusters, but just like, say, with species, culture has two levels.

So if you think about species, There are features that a species shares, every member in a species needs to be able to mate with everybody else. So that means they can't vary that much in a species because they all have to mate with each other. And so there are shared features in a species that are stuck with the species.

And within a species, you can vary all the different things that are allowed to be different in the species. You can vary those much faster than you can vary the species itself. But we have speciation, i. e. new species are created and there is selection of species. And in fact, Places where there are many smaller species have better overall evolution in the long run.

So actually it's more important to have evolution of species than to have evolution within species, interestingly. And the same seems to be true for cultures. If you think about corporate cultures, which is a nice, easy thing to envision, there's evolution within a culture, within a corporation. And so the bigger the corporation is, the Farther anyone's evolution on the corporation can go until it hits the boundary of the corporation where it's harder to spread beyond it.

So for within corporation innovations, a few big corporations would actually be better, but there's innovation of the features of a corporate cultures that are shared across that corporate culture, say, Norms of documenting things or whether you're allowed to disagree with people or, you know, how long people should stay in a position, those norms of a firm, if they're not allowed to vary within the firm, the major way those will evolve is by firms being born and dying.

And it turns out the evolution of firms matters more than the evolution within firms. That is in industries where there are more smaller firms, they just have more overall innovation, even though there's boundaries to things. So then if we get to macro cultures, we have the same lesson. There's evolution within a culture and there's evolution of the culture and the evolution within the culture, the bigger the culture is, the faster that goes.

And that's what we're getting today. Because we have a few big cultures in the world and we're doing evolution within the cultures by investors. Now, like three centuries ago, most of the world was broken into hundreds of thousands of little peasant cultures. With a few thousand people each, and they were near the edge of subsistence, they were poor, there were diseases, there were wars, and so if those cultures went wrong, they just died fast and were replaced by neighboring cultures.

We had strong cultural selection a few hundred years ago, and up until then, we had had this long period of cultural evolution where the features of cultures were pretty adaptive. They were pretty appropriate for their context. And then, here's the scary part. That's all summarized in the next few sentences.

We broke this machine. We have a sports car and we broke the steering wheel. We broke the steering system. It's still going fast. We just can't steer it anymore. What's that mean? A cultural evolution system has the following key parameters. There's a variety of cultures at any one time. There's the selection pressures on any one culture.

How far, if it deviates, will it just get killed? And then there's a drift parameter. Just how quickly does the culture just randomly go in different directions? We've made all three of those parameters worse in the last few centuries, much, much worse. So first thing we made nation States, we took hundreds of thousands of peasant cultures and we smashed them into a few hundred national cultures.

Then in the last centuries, we've enticed elites around the world to form a world elite culture where they get more out of identifying with the rest of the world, at least than they do with their particular country. And we are vastly reducing the world 's variety of culture, even. Much less than a few hundred nation states, we have vastly reduced selection pressures.

The main thing left is just population, but very few wars, few diseases, little famine, we're rich and comfortable, cut down the selection pressure. And the third is we have way increased our drift rate. So you probably know that ancient civilizations were not that eager to change themselves. They were traditional.

They had long standing traditionals. They liked it and they weren't very tolerant of people who wanted to change them. And then somehow in the last few hundred years, we went the other way. We decided that the people who cause cultural change are our heroes. They are our great saints of our world. We have holidays named about them, et cetera.

They are heroes of our novels and stories are the people who cause cultural change. We've all heard the great stories that culture used to be different in the past and now it's changed and that was wonderful. And the people who caused those changes are our heroes. And we have very rapid cultural change.

Which from the inside seems great because we feel persuaded that they were good changes. But if you step and look outside as a system of cultural evolution, we have very little reason to believe these were adaptive changes. They weren't the result of selection pressures or some pre existing program that was executed based on pre adaptive strategies.

These are just random changes. I'll call them moral fashion. So the net effect of these three trends is we broke the steering wheel of a sports car. We are now hurling fast into unknown terrain where we're not controlling where it's going and that's just a recipe for disaster. That's a longer description of, that's the problem.

Steve Hsu: And I mean, is there a direct link between this and fertility? So, so there's sort of separately an empirical case that and in fact, in another session, the Collins's were making this claim that just looking at the empirical numbers, it's pretty clear developed countries are in for, you know, sort of fertility doom and it's almost irreversible in many cases.

Is that directly related to the argument that you've made? Go ahead.

Robin Hanson: So this is, I came from the other direction. I noticed there was a fertility problem. Yes. And I said, gee, that's weird. I mean, I knew that it was weird before, but I wasn't so concerned until I heard about that. Innovation going away.

And I go, okay, this is a real problem. And so I was willing to go back and study it more and say, okay, this is weird. Why is fertility falling looking in different theories? And I've considered a wide range of evidence. And then I realized that this cultural drift problem is just a directly plausible explanation.

And in fact, it's my best sort of general explanation. That is, there are many reasons why fertility might fall, but the key thing is that with a lot of cultural selection. Those reasons wouldn't have been allowed to play out the ones with falling fertility would have just been killed off and replaced by the others Selection pressures would have just maintained high fertility Regardless of what in a the tendencies we might have to get below fertility, but we turned off those selection pressures We vastly reduced variety up to the drift rate and that predicts that we're just going wrong in lots of ways All sorts of cultural norms are going wrong, but one of the key sets of cultural norms is the norms that promote fertility, and if you, if they go wrong, they will fail to do that, and plausibly that's what we've seen.

And plausibly this has happened in the past because the Roman Empire had falling fertility, the Greek Empire did, and plausibly they had a drifting culture as well.

Steve Hsu: So for the audience that isn't that familiar with fertility statistics, what's the quickest way to give a kind of stylized picture of what's happening in Western countries and why do you think it would be very hard to reverse in the next generation or two?

Robin Hanson: The world number of babies per year peaked in 2016 and it's been going down and it's on track to just keep going down indefinitely. And if you look at world fertility rates, they vary quite a bit, but not that much. There's a really consistent trend that as nations have been getting rich, their fertility has been falling.

And that's just, you know, going down those places like India just fell below replacement fertility, the entire South America and Central America is below replacement fertility. People don't quite realize, you know, in the U S we used to think if you wanted to adopt a kid, you'd go to Central America and adopt a kid because they had lots of extra kids there, but no, no more, they don't have extra kids there, we're not adopting from there.

so the world people haven't really kept up often that the world is just following a population and. There's no, the UN has had projections on these things for a long time, and it always made the modeling assumption. It still kind of does that fertility would fall until replacement level level and then stop there.

It just, it never does. They just keep falling because this is a cultural thing. It's not, people thought it was some sort of biological equilibrium thing that, you know, we would adapt and, but no, it's not a biological adaptation. It's actually culture. And these things are really strong if you look at these proximate cultural causes.

People feel strongly about that. They really like them. They are not really very willing to give them up. That's the key thing to see. This is going to keep going down for a while. If you think about the kind of fixes we can propose, people are often like, but that would mess with these cultural things. We don't, we don't want to mess with that.

So yes, the world varies a lot, but not as much as you might think. Like I said, we're half of this world elite culture. And so the world has less cultural variation than you might think. Here's a way to think about it. There are many aspects of our lives. And in some aspects of our lives, we have more variety, like fandom or foods or, you know, clothes or music, right?

In particular areas of our lives, we've vastly increased variety there, like, but there's very little correlation between whether you're a Star Wars fan and a Star Trek fan and how you live the rest of your life, how you deal with work, how you deal with family and children and death and et cetera. All these fandoms are pretty much the same in terms of how they live.

But what we need really is variety as in fundamental values, not variety and all these little things. And I did some Twitter polls where I asked about 12 different kinds of varieties in the world, whether you wanted more or less of those varieties. And the one people wanted the least variety about was fundamental values.

People hate the idea of varying fundamental values around the world, but that's the kind of cultural variety that's necessary to get varying behavior. And that's the thing we're missing.

Steve Hsu: Got it. Jim, are there any Questions related to this that are coming from the audience.

Audience member: Are fromnaming fungus.

I'm sorry if I mispronounced that naming says Robin has become very concerned with population decline. I know you, Steve have talked about about it in the past, but I think he might be interested in hearing the perspective on how China will cope with its fertility rate and then naming says that you are one of the least pessimistic people I've heard

Steve Hsu: So there's a conflation of a couple different things here.

I think China does have a demographic problem and if they don't solve it, eventually they're going to be in trouble. Now, maybe they may be in trouble with all the other developed countries. If it's a generic trend, the specific thing that impacts China 's competitiveness over say the next 10 or 20 years, which is often overlooked by people who invoke demography, is because China used to be pretty poor if you go back 30 years.

only a few percent of high school graduates, maybe at most 5 percent or 4 percent of college graduates could go on for college level education and training, whereas that number now is 50 or 60%. So for every old person that retires today in China The person who's coming into the workforce is typically far better educated and trained.

So if you, if you do an aggregate make an aggregate look at the human capital pool that they have to say, develop new EVs or. Satellites or whatever it is, their human capital pool will continue to get stronger and stronger and stronger for about 20 years. After that he knows the bottom could fall out if they're only, if their TFR is one or something like that.

But that's not a, that's not a claim about the long term fertility problem in China. It's just a claim about what their human capital pool will look like over the next 20 years.

Robin Hanson: A big problem with this topic is 20 years from now, you may remember this conversation say those idiots, they were all worried.

Things are, things are fine, right? But we're not talking about 20 years from now. We're talking about 40 years and a century from now, that's the time scale on which this problem is going to play out. So yeah, there is no particular immediate rush, but it's also not very clear that we can do much, even if we, in the long run, like emperor Augustus of the Roman empire knew they had a fertility problem, tried to institute pretty draconian policies to deal with it and it failed.

France was the first place in Europe to have a declining fertility. They knew how they had a problem. They knew that France would decline and influence if they allowed it to happen, but they couldn't prevent it from happening. In 1905, the sitting U S president said the biggest problem of the country in the day was that Catholics were outbreeding Protestants and they never fixed that.

So we have a long history of these problems being known for a long time and just not getting fixed. So that's pretty scary. Okay.

Steve Hsu: Good. Is there another one in this topic or can I switch to something slightly different? So I, I, before I fully open it up to questions, I had another topic I wanted to raise with you specificallyRobin, which is this idea of a polymath.

So Every now and then, someone will come up to me and say, Wow, Steve, you're really a polymath or something. And I'm sure you hear that all the time. maybe you could just reflect on that. What is it about you that makes you a polymath? What's good about it? What's bad about it?

Robin Hanson: So I think most humans, their intellectual style naturally is to be pretty broad.

And one of the things that academia requires is that you narrow yourself. And honestly, most smart people, People who fail to become academics fail because they fail to narrow themselves. That is, they indulge the spread.

Steve Hsu: Yeah.

Robin Hanson: So if you and I are polymaths, we squeaked by going against the usual advice and it's kind of lucky and managing to be polymaths, even though that's kind of punished.

Right. So, but that makes us proud of it. And there's a sense of which being a polymath is kind of impressive. And that's the danger is that we would want to be polymaths and celebrate it just because it's impressive or hard. The better question is, does it help? Does the world intellectually progress faster or slower in what ways because polymaths exist?

So if we're going to hold it to that standard, I might say, well, there's a niche for a polymath. Not everybody should be polymaths, but maybe there's a niche, but what, what is that niche? What, how do you, how do you use that niche? And I'd say, well, the polymath niche is where you look for things from some fields that you could connect other fields, tools or questions or, or examples of explanation.

It's that sort of cross connection that's the payoff, and there's a cost to it. I mean, it costs more to learn many different fields than to, like, stick in one. So, this is the path we should be looking for and judging ourselves against. If you can't find these connections and make them actually payoff, then no, you shouldn't be a polymath.

So, for example, I typically hold myself to the standard of, if I'm going to go into a new field, I need to stick with it long enough to make a contribution, like I can't just be a dilettante reading about it and checking off stuff I've read. I have to learn enough, figure out a problem, solve it in a way that's an original contribution of literature.

So that's one standard. But even that, you might be just better off staying in one field. but the one thing I'll note is the strategy of collecting different things, you know, in different fields, it scales with time. The longer you apply it, the better it works. So polymaths should peak later in life.

Yes. So, I think I'm still at the peak. I'm 65 years old and I still think I'm at a polymath peak. There's a trade off between collecting more tools and of course, yeah, I'm old and I'm getting slow. But, I still think the more tools are beating out the slow thing, at least for now. And that's what you should hold yourself to as a polymath.

Make a contribution and then expect this increase. So it's a lifelong strategy, right? You're not going to pay off that well as a polymath younger on, right? You don't have that many things you're combining. So the odds you're finding something are kind of low. You'll have to do it for other reasons.

Maybe you just get bored or you have, you know, accidentally thought something was the thing and you go on. But later in life is when you should peak. So if you're gonna stop, some people, you know, when they get tenure, they just stop. If you're that sort of person, yeah, don't be a polymath. That's a waste.

Come on.

Steve Hsu: Right.

Robin Hanson: You have to be ready to keep working hard toward the end of life if you're going to make the strategy work. That's the payoff.

Steve Hsu: Now, Vernon SmithI don't know if I would call him a polymath, but obviously super accomplished in economics. One of the things he said, though, about him being able to introduce new concepts is that he really didn't, if, if, if other people were disapproving of him, like your colleagues are saying, Oh my God, why is Robin worried about that?

Like he spent the whole lunch talking about uploaded humans or something. And we've got the fed interest rate hike to talk about or something. yeah, Vernon Smith would say, like, I didn't feel that I didn't, I didn't feel pain from those attitudes of those other minds around me. How, how do you, how

Robin Hanson: How does it affect you?

So definitely being autistic and oblivious helps. Yeah. They're disapproving, but they don't want to be too obvious about it. So they're kind of subtle and you just don't notice. Yes. Okay. so that helps, but I also think it helps that we can't not care what other people think. I don't think that's an option.

Many people think I just don't care what, and I'm not going to, that's just not true. But. You somewhat get to pick

Audience member: who

Robin Hanson: you care what they think you get to put in that slot some ideal of yours You know, I'm gonna put Einstein there or somebody of that caliber Yeah, if I can put in my slot of the person I'm trying to repress somebody who would in fact be willing to do different things Then I'm going to be much more willing to do that because, you know, they're going to kind of look down on me if I, if I don't like do what seems to me the most promising, right.

You know, Einstein's going to be shaking his head and say, no, no, no, you lost your potential players. Right. So I'm going to be impressing him. Then the urge not to be shamed by him will outweigh these other subtler cues that I miss.

Steve Hsu: Yeah. You know, you spoke of Einstein. So there's this guy, Szilard, who started out life as a physicist and was a professor.

Close friend of Einstein's. And they're the ones who wrote the letter to Roosevelt to get the Manhattan project started. And most people don't know that after the war ended, Zillard went off into molecular biology and spent the rest of his career there. So that's another example of a polymath. He even wrote a science fiction story about talking dolphins, I think, at one point.

okay. Let's open it up to questions.

Audience member: Well, we have actually gotten one in on that. Okay.

Steve Hsu: Let's go back.

Audience member: So, politics for pie guys. How does Robin propose to defend decision markets or deductives from insider trading?

Robin Hanson: Okay. So insider trading in corporations is where officers of the company know things that other people don't know about the company, and then they trade in the market to profit from their insider information.

and then of course, that's the game. Now it used to be completely legal up until 1910 or so when they passed an insider trading law. It was understood as one of the advantages you got from being insiders of the company was that you would learn these things and that was part of your compensation.

And, it would make sense to let a company choose this fact about that company for the following reasons. There is a trade off. The gains that you're getting as an insider from these insider trades are coming from other people who invest in your company. And so when they anticipate, That they will be trading against insiders.

They will be a bit less willing to invest in the company. And if the company is trying to raise capital, that will be a cost. It'll raise their cost of capital. So the company then has a trade off. We want to reward these insiders and improve the accuracy of our stock price by allowing insider trading. Or do we want to prohibit insider trading because we have contractual relationships with all these insiders and buying a contract would be required to limit that in order to encourage more capital and lower our cost of capital.

So. That means it looks like it's something the company should choose. Do they want to allow our insider trading or not? Unfortunately, we banned insider trading for everybody, which is not the efficient thing because some companies might want to have continued to allow it. But in any case, in prediction markets, this trade, this conflict doesn't show up unless the prediction market is about a company where the insiders would then like to take advantage from investors.

But. See, it's taken from investors. That's the problem in a prediction market. There aren't investors. It's a betting market. And we create the prediction market in order to learn things about the company. If the main purpose is to get more accurate prices, then basically we want as many insiders as we could.

Insiders is great. Please, please join the, you know, that's what we're trying to do. So insider trading, it's just not really an issue for prediction markets. Now, manipulation, there's two kinds you might mean. One is where people One to influence a decision that a price will, will influence by manipulating the price.

That is, they're trying to move the price without actually having information. And turns out that people who do that on average lose their trades, even if they could move the price. And other people who know those people are trading in the market want to come in and trade against them. So the existence of manipulators induces the existence of more contrary traders and actually on average increases the accuracy of the price.

So that's manipulation. If you're worried about sabotage, that's about changing the world in order to win that. So for example, say we had a market on a deadline for people who are working on the project, if they could bet against the project making that deadline and then cause the project not to make the deadline, they could make money on their forecast.

So that's a concern in general that you might induce sabotage. A simple solution there is just to make sure that anybody who could, you know, sabotage the product. Only is allowed to have a positive stake. So you give them each 100 if you make the project. You say you can bet this up or down a hundred dollars, but not below zero.

You're not allowed to take a negative. All your information can be revealed through lies and cells that maintain your positive state. So that's a simple fix to sabotage. So in any case, we have simple fixes to sabotage. Manipulation isn't a problem. Insider trading

Steve Hsu: put a lot of money on Biden and then assassinate Trump.

But maybe that's not, yeah,

Robin Hanson: It turns out that's already illegal.

Steve Hsu: Yeah. Okay. Let's do something totally different. Is there a question on a totally different topic? Yes. Yeah.

Audience member: Diana. Does Robin make any life decisions in the hopes that someone will in the future? I think it's

Robin Hanson: a nice side effect perhaps.

So I have this book on brain emulations and I hope that the world of brain simulations will occur. And a side benefit is if I wrote a book on their world, they're more likely to want to bring me back to be part of that world. Maybe let me persist in that world. That's as far as I would get from the simulation.

I, I, I just, I don't actually believe that simulations will be a very large percentage of the future creatures who exist. They will be a small percentage, maybe 1%. So I would rather do things to influence creatures who are the other 93 percent at a 100 to 1 ratio rather than doing things to influence the possibility of one of these 1 percent simulations that, you know, that, I'll do something, but payoff there is much less.

Steve Hsu: So I've been meaning to ask you about thisRobin. You're, you wrote a whole book on it, so you're obviously willing to entertain the idea that there'll be lots of artificial minds, maybe sentient ones in the future. Thank you. Does that increase your probability that we could currently be in a simulation now?

Like, what do you think are the odds that we're in a simulation right now?

Robin Hanson: So I have thought about this issue a lot. I have a paper called How to Live in a Simulation, describing that the possibility that you might live in a simulation should change your life strategies a bit as a function of that probability.

But on reflection, I still think it's unlikely that we're living in a simulation. It's possible. And the key parameter is How fast does interest in the past decay with distance in the past? You see, so in the future, the economy will grow and then in a larger economy, they'll have a larger percentage of it doing simulations of its path.

So the number of simulations I'm in is this product interval of the two functions. One is the function of the growing size of the future multiplied by the diminishing interest of the future in the past. And so the key question is, does interest in the past diminish faster or slower than the whole economy grows?

Yes. So if interest in the future diminishes slower than the economy grows, we get this increasing interval into the future. And yeah, I'm probably in the simulation, but if interest in the past falls faster than the economy grows, then most of that interval out there is, you know, they're mostly simulating people more recently in their past.

And I'm too far in their past for them to matter. We can test this. Because you can just type it in a year into Google Ngrams and see all the mentions of a year around a year. And then you can just see how fast interest in a year decays after that year. And it's faster than the population grows.

It's faster than the economy goes. So I therefore conclude that, plausibly, there will be many simulations in the future. Future people should maybe worry about that more, but I shouldn't worry more about it. Because basically by the time they can make simulations of their past. I'll be too far in their past to be of interest, except maybe Diana has a mind since I wrote the book on the concept of brain emulations.

Maybe I would be of special interest to them in simulating. So maybe I should adjust my calculations for myself. But

Audience member: correct me if I'm wrong, Steve, but I think you are. you think it's very plausible.

Steve Hsu: I think it is plausible. I think more in terms of the integrated overPast and future number of wetware evolved beings that are above some level of sentience or intelligence versus the number of virtual ones.

And I think one parameter, which maybe isn't in the toy model you just described, is the cost, ease with which, you know, with which a five year old AI kid could create like an incredibly large simulation full of beings, every one of whom is more sentient and smarter than you and me. And if that cost is very low, then there just could be just tons of that stuff running.

And then. Of course, there's no, we never know the a priori probability measure over all this stuff. Right. Because we're placing our consciousness somewhere in here. But if you had a flat probability distribution, then it could just be overwhelmed by the ease with which they can make crappy brains like ours.

Robin Hanson: Right. So in Age of M, my book on brain emulations, The key observation is that when it becomes cheap to make brain emulations, then there become a lot of them. And it's a Malthusian world where most of them are near subsistence.

Steve Hsu: Yeah.

Robin Hanson: And near subsistence brain emulations cannot afford to create many more of them in some sort of five year old toy simulation, right?

The five year old AI you're imagining has to be really rich. You see, compared to the median income in order to afford to make vast numbers of creatures, right? I'm assuming

Steve Hsu: some kind of world of super plenty, you know, where the AI said, I'm not assuming the world.

Robin Hanson: I'm assuming a world near subsistence at a Malthusian equilibrium.

That's the analysis I would defend. Got it. The future is likely to be Malthusian. Oh, that's depressing.

Steve Hsu: next, next topic.

Audience member: Well, there was one more, and maybe we've covered it all. Sam just said, have you updated on the scenario based on recent generative AI or research in the fertility population?

Robin Hanson: But maybe we've covered it all.

Well, certainly, I was presuming in the age of M that we would continue at roughly exponential growth, and that's when I would project the age of M would come. If we're going to face a several centuries long innovation pause, then it'll have to be played after that, unless it can happen before, which I think is unlikely.

So, yeah, there's a long pause there. The other part of that was, sorry.

Audience member: And recent

Robin Hanson: research in the population for that was about AI. Sorry. Right. So, obviously, you know, recent developments in AI have been exciting and, you know, they are a fluctuation in long term trends in the sense that, you know, the history of AI is 70 years, at least of developments.

And that hasn't been exactly equal developments. There have been moments when new developments appeared more dramatic than in the previous year. And, you know, last year was a year like that, but I still think it's within the range of ups and downs we've seen over 70 years, so I wouldn't update on the overall distribution.

I'd still say the related thing was 9-11. On 9 11, right afterwards, many people said the distribution of terrorist attacks just changed.

We are now about to see many more, much larger terrorist attacks. And other people who were right said, no, you just got on a usually high draw from the same distribution we've been drawing from the whole time.

I would say the same about AI. We just got an unusually high draw from the same distribution that will continue into the future. AndI don't see evidence to the contrary. Neither should you have thought that in 9 11, right? It's not enough just to have seen an unusually high draw. You'd also have to see a process that would cause and continue to make more unusually high draws like that.

I didn't see that then, and I'm not seeing that here now.

Steve Hsu: Well, we could get into that a little bit. Soto Jim, do you want to make a comment since you're here? You might as well.

Audience member: Yeah, I know. I was just going to ask, like, I'll put my own spin on this for a second. I can imagine in the future, if we are about to go through the singularity, if I could imagine in the future, one of the most simulated events would be the singularity, one of the most important historical events ever.

So if we witnessed the singularity right now, what are the chances that out of the maybe trillions of times that that actually is. simulated that we're observing the real.

Robin Hanson: By singularity, you mean a sudden, very rapid increase in growth rates that very quickly leads to a very different world, right? Yes, plausibly, people like to simulate pivotal events in history more than other events.

And in fact, that's typically what we do in any simulation. So if you imagine you simulate an airplane wing or something, right? It's flopping around. We have a grid and we make the grid more fine whatever things are changing more, not just in space, but also in time. So it's just a general fact about simulations that you prefer to simulate pivotal events in more detail.

And therefore, yes, you would simulate this if it was about to be a pivotal event, but I'm just not very persuaded that that's how AI is going to play out is a sudden thing. And then in the next two years, everything changes. I

Steve Hsu: I think I'll share a story that is relevant to this topic, which comes from impeccable sources.

so several founders of open AI were present at this discussion. and it's about the creation of open AI, which goes way back because it was first created as a kind of not for profit thing. Elon Musk became concerned. That, Elon actually has a fair amount of probability weight on us living in a simulation.

Right. it's easy if you're the richest man in the world to think that More

Robin Hanson: plausible.

Steve Hsu: This is a simulation, and I'm one of the player characters, and you guys are mostly NPCs. Right. But there are a few other player characters, and what is the point of the game? So, he sort of became convinced that, and this is at a time when DeepMind was making a lot of progress, AlphaGo is his time.

Right. Right. And, he became convinced that, yeah, he became convinced that maybe it was Larry, but even more likely Demis was another player character. And the, the, the point of the game was to get to AI AGI. Okay. And that is actually why he put in the money to create an open AI, because he was afraid that Demis was going to get too far ahead.

And this is actually, I think, a completely true story. It's widely believed by people who are close to the events.

Robin Hanson: I mean, I think it's psychologically pretty healthy for people to have grand ambitions in the world and see themselves as important and see themselves as doing pivotal things. It motivates you, it gives you energy, you focus your attention.

So I'm happy to see people seeing themselves as part of big, important things, but still from a distance, I can't. You know, have to ask, yeah, but is it true? Yeah. And, you know, and I've seen, you know, all through my lifetime, lots of technology people who are really excited to believe that they are part of pivotal things.

And that's very energizing. You know, in some sense, you can't really do a startup unless you kind of irrationally believe too much in it. Right. So. And Musk is doing wonderful things. I want him to believe wonderful things about what he's doing. He's one, you know, one of the greatest people in the world today in terms of an accomplishment he's thinking about.

So I will give him all the delusions he wants about his pivotal role and the things he's doing in it, because a lot of them do great things, but still I got to stand back and say, yeah, but what's the chance.

Steve Hsu: Good. So let's, for these wonderful people who are actually here physically at Manifest, let's give them an opportunity.

Does anybody want to raise your hand and ask a question to Robin? Yes, go ahead.

Audience member: Sure. Yeah. So regarding cold trends, it seems like your main concern is finding innovation rates. But if you look at who actually innovates, it's a very small fraction of the population on the right

Robin Hanson: side of

Audience member: the distribution.

Yeah. you know, for everyone. Point drop in IQ, you get a 20 percent drop in the right tail, 140 plus. so, anyway, there's a lot of gene editing tech, which I think is much less speculative than AI. You know, there are startups today doing it. So, trio selection and so forth. How could that maybe play into this thesis?

Could it counteract some of these concerns? You know, you have a lot of. Professional class people creating super babies, as it were, right?

Robin Hanson: So as economists, we study innovation is one of the most important things we study, and we certainly see a lot of variation in rates of innovation. Small scale. We know it's the most important thing going on, and many politicians and government officials know it's very important.

Excuse me. And so for a half century, at least, many great places around the world have been trying to become the next Silicon Valley. They say, yes, yes, that's very new. We should be able to do that too. We have smart people here. We have technology, we have buildings, we can be the next Silicon Valley. And so they thought, see, it shouldn't be that hard to increase local innovation rates.

Look, they're doing it and they mostly fail. So it turns out it's hard to just try to cause higher innovation rates locally by hopping to Silicon Valley. It's a. It's a complex cultural thing. Silicon Valley is successful because of opaque cultural things that are hard to copy. That's part of the nature of culture.

So yes, innovation rates vary enormously. And if you could turn knobs to turn them up, why then sure, small in population, we could compensate by turning these knobs and turning up innovation, but we don't have these knobs. Now you say, ah, but we're going to make some smart people. And I'll say, It's not so obvious by just making a bunch of smart people.

You can make another Silicon Valley. Sorry. That's, you know, there's mostly cultural things. You might say, oh, we'll send more smart people to Silicon Valley. I'll have more hope about that, but I'm still not sure if the number of smart people is really the limiting factor on Silicon Valley being bigger. I don't think so.

I think there's plenty of people around the world. We could send Silicon Valley to make it bigger. If we had a way to do that, we don't know how to make Silicon Valley bigger, even by sending smart people there. So I'd say. You know, I'm not that hopeful, but I'm happy to let people try and crank up the IQ clock.

And maybe that'll help do it. Go for it.

Steve Hsu: Last night many of us were at an after party at Menchus mole bugs house. Anda story was told of a startup that I had never heard about called leverage. Apparently leverage was funded by Peter Thiel and it was actually a kind of psychology. Project to try to study founders and try to understand the psychological profile of these founders with the goal of creating more founders.

And, if of course leverage, I think didn't succeed actually in, in what it was doing, but had it succeeded, maybe you could gene engineer founders, not just smart people, but people with the right combination of IQ and drive and narcissism and whatever it is that Eagle mania, whatever it is, you need to be a good tech founder, but, but still, it's still not an imminent technology that So, okay.

We're almost out of time. Any, any last question? Yes.

Audience member: Just on the conversation of you both being polymaths, I'm curious to know if either of you consider that there was a counterfactual for both of you, where you were specialists, and what that future for each of you respectively would look like, and how you would feel about that.

Steve Hsu: I can answer that because for the first part of my career, I really was a specialist. I really wanted to be a particle theorist, a theoretical physicist. And that's what I did. And I guess my pattern, I don't know if it's the same as a Robin's or not. He can comment, but my pattern is I'm kind of curious about everything or a large chunks of stuff.

And I want to learn stuff. And I actually have to suppress, like stop myself from going to learn other stuff. And so I sort of had a specific strategy for myself when I was young that I wanted to be my hero, Richard Feynman. I want to make a contribution in theoretical physics, and I'm really going to focus on that.

And I'm going to consciously turn it off. Some of my outside interests, which could be intellectual interests, or it could be like hitting on girls or whatever it is. I just suppressed that down a little bit so I could at least succeed in the chosen narrow specialty. But then as I got further along that tenure, things like this.

I could say like, okay, now I'm going to spend 10 percent time on computational genomics, or now I'm going to spend more time on AI and machine learning. So, for me, it was a very conscious kind of negotiation of all that stuff.

Robin Hanson: I'd say the non polymath counterfactual of me is much less plausible than the academic failure counterfactual of me.

That guy would continue to be a polymath in his free time on the side. I'm pretty sure he would have continued as a hobby polymath, but he would not be an academic. He would be a software engineer, say, and spend most of his time doing the work and then on the side being another thing. That's the, by far, more common human pattern.

I think I was going to be a polymath one way or the other. In my free time, the question is, could I get a job where I could be a polymath? That's the harder part.

Steve Hsu: I'll say something which is a little bit elitist and, you know, when this goes on YouTube, people will be mad at me. But there are many people who in a way are like, just below being polymaths in the sense that they have broad interests and they're learning stuff all the time, but they're not really able to make contributions in those areas.

I would almost say, whether it's a matter of high concentration or drive or super high intelligence, what the quote, real polymaths are the ones who they can dabble in something and they actually make a contribution. And that is extremely rare. So,

Robin Hanson: right. Yeah. Right. And that there's a sense to which, I think polymaths will more just have a self sense of direction and decide what to do in a way that most academics, even most people, they mostly go along with what the environment around them tells them to do.

They don't have to invent their own ways of doing things. There's just an established procedure and they just learn it and they follow and they just do it. So most people don't have to invent themselves. They don't have to invent their priorities, their strategies, or their whole thing. But if you manage to invent yourself.

And as if you manage to become a person who will decide what you do and figure out your strategy, et cetera. Now you have more of a shot at being a polymath because you will have to invent that more. and I think that's a thing. I didn't realize what I was doing when I was young that other people didn't do.

And that's opened up many things for me. Also closed doors as well. Butyou know, as an undergraduate, I just, as I mentioned, like, stop Doing the homework and just study things by myself. I was inventing how to study it. I just made up ways to study. I didn't follow a curriculum and then I made up new projects for myself and pursued new projects for myself.

And it seemed obvious at the time that, Hey, I was interested in this. Let me just do it. But a lot of people won't do a project unless they're shown how to do the project and have a script for doing it. And they won't, you know, and then they learn in their field, just follow the scripts of the field. And there's a sense of which it's worth trying maybe early in life to just Be on your own and just try to do things yourself without scripts and without so much advice, because if you can like do that a while of it, do it okay, then maybe you can actually do whatever you choose, including make a contribution.

Because many people basically, you only know how to make a contribution to your field because people showed you the script for making a contribution to your field. And that doesn't work. If you go dabble in another field, you don't know their scripts for making contributions. You have to make it up. Okay.

Yeah, I think the

Steve Hsu: whole, we'll get to your question in one second, and you'll be the last one. So, I think one of the really amazing parts of there being a startup culture, and pools of investment capital that are willing to bet on young people with ideas, and valorizing the idea of going off on your own and doing something, is it creates the potential for many, many more polymaths.

So, I think we're going to see a lot of people who I did my first startup with. I dropped out of college at my first startup. I was successful. I'm going to do something totally different now for my next one. And so there's a nice machine for innovation and creativity that now exists that totally did not exist when we were young.

So it's great to see it. Last question.

Audience member: So Robin, we're facing a decline in fertility where the population and additional ones will also have a drop in innovation. I'm interested if you were asked why either of these did not happen, like, let's say either, either there is a collapse in fertility, but innovation does continue, or if there is not a collapse in fertility, why would that happen?

Robin Hanson: the first one's the second one's easier if you say, why would fertility not have fallen? We have a number of possible policies that people have thought through, including myself, that would work to revive fertility if in fact we were willing to adopt them, such as, for example, giving parents a transferable percentage of their kids' tax pay.

It's the future. A large percentage. I think that would just work. It would cause cultural changes. Many people wouldn't like it, but if we were willing to stick to it, it would work. So I would say the most likely scenario, if the mainstream cultures revive fertility is because they will have adopted one of these things we know work, but then you say, what if fertility falls, but innovation doesn't fall?

Well, now we need one of these stories where somehow we've disconnected innovation from the population by somehow vastly increasing the number. Percentage of people who are innovating, right? And or AI, for example. Well, obviously if AI, a full human level, AI is equivalent to not falling fertility because we just have full replacements for human labor in the form of AI.

So that's another way in which we might not see a falling population if we just have AI, in which case we can have plenty of AI doing the innovation. Right. But, it's pretty hard to just vastly increase innovation without increasing the population, because that's what nations have been trying to do for a long time.

A lot of places say, let's increase innovation here. They would really love to. They're hoping for lots of money and rewards from that. And they just keep failing. So it's hard.

Steve Hsu: All right. Well Thanks everybody for coming in, especially thanks to Robin for spending time on hosting. All right.