Pondering AI

Stuart Mills scrutinizes the value of constraint, the aesthetics of expertise, our mythical quest for the perfect decision and how AI innovation stacks up historically. 
 
Kimberly and Stuart discuss the history of behavioral science; what we can learn from bounded rationality; whether AI degrades human expertise; the value of knowledge work; science as a social act; challenges to public knowledge and expertise; the power of behavioral science; the value bias problem; intuition and discernment in decision-making; human discretion as an antidote to friction; the comparative impact of AI-enabled transformation; and our actual appetite for radical change.  
 
Stuart Mills is an Associate Professor of Economics at the University of Leeds. A prolific researcher and writer, Stuart's broad interests coalesce around AI, behavioral economics, and the political and digital economies. 
 
Related Resources: 
A transcript of this episode is here. 

Creators and Guests

Host
Kimberly Nevala
Strategic advisor at SAS
Guest
Stuart Mills
Associate Professor of Economics, University of Leeds; Visiting Fellow of Behavioural Science, London School of Economics and Political Science

What is Pondering AI?

How is the use of artificial intelligence (AI) shaping our human experience?

Kimberly Nevala ponders the reality of AI with a diverse group of innovators, advocates and data scientists. Ethics and uncertainty. Automation and art. Work, politics and culture. In real life and online. Contemplate AI’s impact, for better and worse.

All presentations represent the opinions of the presenter and do not represent the position or the opinion of SAS.

KIMBERLY NEVALA: Welcome to Pondering AI, brought to you by SAS. I'm your host, Kimberly Nevala.

In this episode, we're joined by Stuart Mills. Stuart is an Assistant Professor of Economics at the University of Leeds, and a leading researcher in AI, behavioral economics, and the digital economy. He joins us to discuss how behavioral science and behavioral economics are influencing how we think about design and adopt AI systems. Welcome to the show, Stuart.

STUART MILLS: Hello. Thank you for having me.

KIMBERLY NEVALA: Now Stuart, you started out your educational career focused on finance and accounting. And then subsequently switched your focus from accounting for dollars or, in your case, pounds to accounting for behavior. What was it that first interested you in economics at large and then turned your attention to this question of behavior?

STUART MILLS: I mean, the honest answer is I studied finance and accounting, and then I got my degree and became very paranoid that any jobs in that sector were going to be automated away. So I thought, I've got to diversify and learn something a bit more robust. Not to say that finance and accounting isn't a robust discipline. So I went on and studied economics proper. And at the time, behavioral economics, and what became behavioral science, was the hot thing. And I just, I guess I was influenced, I was nudged, we might say and made a career of it, I guess.

The anxiety about automation belies a career interest in technology. So whilst I have an economics background and whilst I moved into the behavioral science space, my focus has always been on the interaction of technology and behavior, and technology and the economy. Nowadays, I have to keep doing 15-second elevator pitches and I can never quite boil down what I do into 15 seconds. But that's the broad blob of stuff I'm interested in.

KIMBERLY NEVALA: See folks, anxiety can be beneficial. So that's fantastic.

And I think we probably all have a sense that we know what behavioral economics or even behavioral science is. But from your perspective and having worked in this from the early years, how do you define it? What are the origins and aims of behavioral economics and its relationship to behavioral science?

STUART MILLS: If you want to look at the history of it, there's a long history and there's a short history. The short history, which is probably the most well-known and probably the most popular, basically goes like this: We have a model of economics that says people are very rational. We maximize all of our decisions. We're brilliant computers and we have utility functions in our head. And then in the late '70s, early '80s, a bunch of psychologists, notably Daniel Kahneman and Amos Tversky, they do a whole bunch of psychological experiments on very typical economic choice problems and demonstrate that lots of the economic models do not conform to how people actually behave.

They make some very brilliant adjustments to the economic models, which is - that adaption of economic models is often missed in this narrative - but it's crucial for how the field actually develops. So behavioral economics, at least a short history of behavioral economics, is this adaption of economic ideas by bringing in psychological insights to create models that are more realistic in terms of describing human behavior.

Throughout the noughties and 2010s, with the popularization of these ideas through books like "Nudge," a broad and modern behavioral science emerges which draws on social psychology as well. Thinks more about norms and group behavior. So it's that building out from this very individual-oriented heuristics and biased model. That's the very short history of behavioral science that most people know.

The longer history, I'll keep this brief, but the longer history basically starts with organizational theory. It starts with ideas about how governments and big organizations should work. Classically, we imagine in our head like a pyramid hierarchy structure. And you have people like Herbert Simon saying, well, actually, no, organizations don't work in these very structured, rational ways. They work in a much more interpersonal, sort of blurry control structure kind of way. And Simon and his colleagues at Carnegie Mellon push for a more empirical understanding of organizations, drawing on psychology and implicitly, then, end up making a critique of economics. But we don't need to necessarily get into the long history. There is a longer history, if you want to elaborate.

KIMBERLY NEVALA: And is it fair to say that a lot of the focus areas, that the logics, or the trajectory, if you will, of how we are thinking about and looking to apply AI systems are animated by or have rested on a lot of these behavioral science or behavioral economics foundations?

STUART MILLS: Yes, but in a way most people I don't think know. So again, if we split it into the long and short history of these two paradigms, two perspectives on human behavior. Right now, there's a whole bunch of interesting stuff about do AI systems exhibit biases, in the sense that behavioral economists think? Like, loss aversion, status quo bias, things like that. And basically, can we understand AI systems in the same way that, nowadays, with modern methods, we interrogate human behavior?

And that's really interesting stuff. And there's lots of emerging evidence that suggests, actually, really advanced AI systems do exhibit human biases, which, to an extent, you'd expect. It's trained on a whole bunch of human data. And so the modern perspective is really we can use behavioral science to understand how these systems work, and then perhaps to think more intelligently about how we can integrate them.

If we look at the long history, and this is the aspect that most people don't know. I mentioned Herbert Simon. Herbert Simon is my intellectual hero. He's also the father of, or a father, of artificial intelligence. He came up with the idea of bounded rationality, which says we're not perfect calculating machines. We make good decisions with limited information. And he said if we wanted to build intelligent computers, we can't build a computer that knows everything. We can't build an infinitely powerful computer. We've got to look at the limited computing resources we've got and use intelligent ideas about how decision-making happens and how information processing happens to make an intelligent machine, given limited resources - be it data or be it compute.

In my opinion, that's a very valuable perspective we should resurrect. Particularly when we're talking about this race for compute. Particularly when we're talking about the exhaustion of data and training these super advanced models. But it's also not the connection between behavioral science and AI, which is often nowadays talked about, because it's a history that is not as popular as the modern behavioral science as I have phrased it.

KIMBERLY NEVALA: So I want to linger on that idea of bounded rationality for a minute then. Why do you think that's important, in the moment that we are at right now? Particularly with the generative AI boom and this idea of all of us getting our hands on, and I'm making quotes here, on all of the information ever known? Insert eye roll. But why is this an important idea for us to think about? And how might, from your perspective, that influence a different, I don't know if it's a different direction or just a different emphasis, in the work we're doing today?

STUART MILLS: I mean, I'll answer the question from a model design perspective, and then from a how humans use AI perspective. I should caveat by saying, so Herbert Simons, it's often called good, old-fashioned artificial intelligence. That paradigm was tried for decades and did not achieve anywhere near the apparent intelligence that current systems have. So I'm not saying a move back to a bounded rationality paradigm is some sort of key to making better AI systems.

The reason I bring it up, from an AI system perspective, is to highlight the fact that we might be getting results, which some people think are impressive. But we're achieving it in an incredibly wasteful, incredibly resource-intensive way. And not a way which necessarily gives us deeper insight not into how the systems work but into how the problems that we're asking the systems to solve actually come about and how those solutions come about.

It touches on things that are becoming deeply political. When we're talking about building data centers to supply the compute, all the water that's required to cool these systems, all the data that's required. And I'm sure you've noticed, I'm sure most of your listeners have noticed, the internet in the past three years has become increasingly difficult to navigate, because everything has to have a human checker involved. That's to deter web scrapers.

And so we're, well, I say we, as a species, we have created systems which are producing outputs which look kind of impressive, but we've done it basically by throwing everything at the wall. And I'm not a computer scientist, so I'm not going to claim that I know some way of doing it more efficiently. But I think the lesson of bounded rationality that actually constrains on compute and information are useful, because it forces you to find smarter ways of cutting through the noise and making better use of what you've got. I think that lesson is something we should return to if these systems are going to become more efficient and if some of the social externalities of AI systems are to be ameliorated.

On the flip side, when we talk about people using these systems, I think it's important to reconnect with this idea of bounded rationality, because - this is very much my opinion - the more modern version of behavioral science, which emphasizes human biases, has implicitly also emphasized we can make better decisions. That if we de-bias ourselves, if we get more information, we can avoid making these errors of judgment. And there is some truth to that.

But it leads to a series of behaviors, I think, which basically we demand everything. We try and get as much information as possible. We throw as many resources at a problem as possible. Because we want to get that perfect decision, rather than that good-enough decision. And AI systems just proliferate this problem. ChatGPT can produce massive amounts of information for you really, really cheaply. A person who thinks in terms of bounded rationality and says, I'm going to settle for a good-enough decision and have confidence in my decision, doesn't need as much information as someone who is obsessed with making the perfect decision.

And when push comes to shove, that perfect decision will not actually be as perfect as that good-enough decision. They'll actually be quite similar, but one person will have wasted a whole bunch of time generating information and filtering through it. So from a designing these systems perspective, but also a human interaction perspective, I personally think embracing this idea of bounded rationality, again, from the positive and the negative, is quite valuable at this moment in time.

KIMBERLY NEVALA: Yeah and, well, and it also perhaps relieves a little of the pressure folks feel these days of having an opinion about everything. Which is not just an AI problem but also being an expert in everything. This sort of idea that you can be that. How do you see this then playing into how we tend to view human expertise these days and how we tend to value that?

STUART MILLS: I think there's a general tendency, we are seeing an implicit degradation of human expertise. AI has made it easier for people to access perhaps expert-level information but also to produce the aesthetics of expertise. Which is necessarily going to degrade the reputation of experts in the long run. We've already seen the social phenomenon as a society, like for a decade-plus with Dr. Google. The internet just made accessing medical information so much easier. And all of a sudden, people started to respect doctors' opinions less and less and there became a much more animated discourse about medicine much more. And so I suspect that in the coming years, we're going to see that bleed into all sorts of other areas of expertise.

I also think, though, paradoxically - how to phrase this? We might start understanding the role of the expert in society as there being a hard cutoff in 2022, where if you were an expert before 2022, you've got street cred. Your kudos are assured. But if you become an expert after 2022, it's not that you're not an expert but perhaps people see your expertise as less reliable or less genuine, or whatever word we want to use. And I do wonder in the years ahead.

And I'm an academic, I write papers, I do research, this is what I worry about. I do worry in the years ahead that the stock of pre-'22 researchers is going to rise, because getting a pre-'22 researcher on a paper suddenly gives it more credibility than a paper written by someone post-'22. And how do experts who have that pre-AI stock, how do they exploit that potential opportunity, that potential human premium. Which comes not through being an expert, but through being born at the right time? Born in the right generation, as the music comments might say.

KIMBERLY NEVALA: I mean, so how does that then influence the folks' desire even to become an expert or to focus on a singular thing do you think? Do we all become generalists? I would hope not.

STUART MILLS: Well, I don't think we all can become generalists. Or to put it a different way, if we all became generalists, then you'd very, very rapidly see a collapse in income levels in the Western world. The value of knowledge work - broadly defined in the US, the UK, Europe, all knowledge economies - the value of knowledge work is in its specificity. It's you know something really, really well, and you know it to a level that's intuitive. And you can spot connections which a non-expert, who has access to all that same information, would never be able to spot. So we can't all become generalists. It would just erode this notion of knowledge work entirely.

And I don't really see the political desire to uproot how the whole Western economy works and go back to manufacturing stuff; whether that's good or bad. So we can't all become generalists. But there is, at least this is the impression I get from my students, who are about 10 years younger than me, there is definitely a rising anxiety that to get to a point of expertise that is actually worthwhile is inhibitive to most people.

I don't know how it is in the US, but I can certainly say in the UK, we're seeing more and more people choosing to do postgraduate study, and postgraduate study becoming much more of a normal thing. I've had Chinese students tell me that in order to stand out in the Chinese labor market, postgraduate studies are becoming more and more of a requirement.

And the further you go, and I've taken it all the way, you get to hyper specificity You get to a point where if someone wants your knowledge, it can be treated as valuable, but it is also inhibitive. It requires a tremendous amount of ability, but also a tremendous amount of funds to go through that education system. So we can't all become generalists, but there needs to be some, in my opinion, needs to be some thinking about the process of training knowledge workers, not because AI is displacing them, but because AI is forcing everyone to step up the knowledge ladder. And how do we make sure that enough people can step up on that ladder? It's a difficult social problem.

KIMBERLY NEVALA: For sure. And as you were talking there and noting that at a certain level of expertise or knowledge, things almost, decisions become intuitive. And it seems to me that this idea that human decisions, that the idea of intuition or the idea of things that we make based on gut feel is, in and of itself, seen as a negative. And it starts to then play into, potentially - and maybe it's a loose connection - into this thread about we should lean more and more into things like AI and AI systems for decision-making, because they are objective arbitrators.

And we, as humans, are capricious creatures who are, in fact, not rational and have biases. And you had a really interesting working paper recently about really looking at or thinking about this, what you call the algorithmic debiasing argument. So what was it that was the genesis for you of that paper and that particular work?

STUART MILLS: Well, I think it's a popular myth to, A, believe that we can make decisions that aren't biased in some way. And B, to think that in an organizational setting, whatever that organization might be, we can actually get to a point where the correct decision is open to us and we're making it in a rational way. Somebody is always acting on intuition in an organization when a decision is made. It just might not be obvious where the intuition is happening.

The most obvious example I can give you is say we have an organization. And every rung of this organization is collecting all the data, processing it properly, feeding it up to the next rung, who are collecting the data, processing it properly. Yeah, yeah, yeah. It eventually goes up to the CEO, and the CEO just has to make the decision.

And there's two ways of playing this. Either the CEO just does what they're told; in which case the CEO never made the decision, the people below them did. Or the CEO has to basically act on their expertise and act on their intuition and take this information and boil it down. Like, based on ultimately what I have to hand, what do I think is best? So there is always intuition in decision-making. We deceive ourselves if we believe we can build systems that eliminate human intuition.

Most often what happens is we build systems which hide human intuition. Which push intuition into areas that perhaps we wouldn't want people having that discretionary power. Or which funnels intuition and discretion to a smaller and smaller group of people. Which, depending on your organization, you might say creates challenges around democratic legitimacy.

So a big impetus for the paper was not that algorithms are bad, or that people aren't actually biased, or that we shouldn't try and improve our decision-making structure. It was to highlight that we do ourselves a disservice and we create hidden dangers if we allow ourselves to believe that we can actually eliminate intuition and all the potential biases which come with it. Intuition always exists. Technology just pushes where that intuition and where that discretion happens; in places which sometimes we don't notice, and often we don't actually plan for or think about.

KIMBERLY NEVALA: Well, and there were actually a number - and we'll link to this paper as well - of, I think, really striking observations and arguments in the paper. One, which I think is probably important for us in this moment collectively to look at, is this idea of - and this, I think, is my words, not your words from the paper - but science as a social act. And in fact, even science as a political act. Can you talk a little bit about that and why it's really important we understand?

And I think, tell me if I got this right, but it wasn't so much about: is the observation we made correct or biased? But it's the interpretation of that observation, or the metric that's produced, or the recommendation that a system is spitting out, the decision that it's making, that is always going to be influenced by the human environment, if you will.

STUART MILLS: That's the long and the short of the argument. The phrase from Levins and Lewontin, which I like, is gravity is the same under socialism or capitalism. They're not saying, in my interpretation of what they're saying, it's not a denial of gravity, it's not a denial of observation, but it's recognizing that our observations can never be divorced from our position as social beings existing within society.

And I find this, as a topic, really interesting and really important right now, Partly because there's so much challenge to this idea of public knowledge, public expertise. Can we trust scientists? Do scientists have agendas? And science is increasingly politicized in ways, which I know, speaking to scientists or people who call themselves scientists, that they don't appreciate and that they find, actually, to be quite regressive. And certainly, AI compounds this problem. If everyone can pose as being an expert because of AI, it degrades this idea of well, what actually is scientific practice?

So when I talk about science as a social act, I'm not trying to denigrate science. I'm not trying to say, well, this is all a mask. It's all an illusion. And scientists aren't actually as robust and free of bias as they claim. What I'm trying to argue, and what I would suggest Levins and Lewontin many years ago were trying to argue, is that by recognizing that science exists within society, that our observations are just observations, but it's our interpretations which affect the world, it can empower the scientists, and it can make the scientists understand that they play a role as a citizen, as an actor in the world. That scientist is not something that they embody and they can't step out of society.

I worry I'm getting very, very highbrow with this or very, very abstract with this, but I do think it's really, really important. And just to tie it back to behavioral science, I think it's really important and this is the argument I make in the paper. Because behavioral science says a lot about decision-making, it says a lot about when people make bad decisions or make errors in their decision-making. And obviously, if someone's making an error, if someone's doing something that's causing harm or creating an inefficiency, we should look at it. We should look at it critically and try and resolve that problem.

But it's an incredibly powerful ideational technology or ideational weapon. I don't want to be too provocative, but it's incredibly powerful, behavioral science. Insofar as if I can label you as biased and I can say, well, Kimberly will make biased decisions, I can start constructing an argument for why you should have less power, why you should have less authority, why your actions should be constrained.

And when we're applying that argument to doctors or teachers or judges, people who serve important social functions in our societies, I think it is right to step up and say, OK, I'm not saying that all scientists are wrong, but we should recognize that what behavioral scientists are saying is not just confined to the scientific world. It has implications on the political and social world as well.

KIMBERLY NEVALA: Well, and you make this point about differentiating between when we think we're observing bias versus observing decisions. Or the outputs of decisions and the different types of bias. And there's some really great examples in there about, in the criminal justice system, about a current offense bias. Like, this is actually a bias, is it not a bias? It really depends on how you look at it and think about and how your justice system works holistically.

You talk about the UK grading scandal during COVID and our predilection, I think, to perceive impartiality, I suppose, in algorithms, and therefore move it. And that, to me, is I think what really jumped out. Because it's really easy, and maybe this is starting to point to a little bit of this question of accountability and responsibility in an age of AI. And in an age where decisions are getting pushed more and more, and there's more and more argument for more and more of those decisions, and consequential decisions, to be pushed to AI systems holistically.

STUART MILLS: And somewhere else, I'm not sure in that paper, but somewhere else, I think I've called it the value bias problem. And it essentially boils down to it's often quite easy to reconstruct a value debate about how should society work? What do we want our institutions to do? To reconstruct that debate over values as a bias, as a behavioral bias, as some sort of error of judgment. And once you turn a value into a bias, I'm going to make up a word, you sciencify it. You scientifify it.

But suddenly, it stops being in the realm of politics, in the realm of the social where your values and my values, whilst they might clash, they're both valid because we're both citizens. And it starts being this: the bias exists as an objective fact in the world. And regardless of what you and I think, we have to deal with the fact that the subjective objective fact exists.

And that's powerful, particularly in a democratic setting, because it allows you to move from debating values and debating things like accountability over decisions, and it's moving towards a more technologically mediated system where, when things go wrong, politicians and policymakers and CEOs and whoever else, can throw up their hands and say, well, we were just following the science. Or, well, we didn't make that decision. It's just what the algorithm decided.

It allows us to divorce these important value debates, which lead to accountability, from ourselves. And to create this false idol, the computer, the scientist, who is actually passing judgment. And that's really, really dangerous. And as you so kindly bring up, my paper raises several examples.

Judges who focus on the current offense when they sentence people or decide to give bail. They might be biased. They might be ignoring all the criminal history, and the criminal history might statistically predict the chance of re-offending. I'm quite sure it does. I don't dispute that finding. But if you believe in the value that criminal justice should be a system built on reform and forgiveness, you shouldn't judge someone for something they did 10 years ago. You should only judge them for their current crime.

So is it a bias or not? Or is it a value? And there are several other examples. If your listeners want to see other examples, they can go read the paper. But I think it's really important to recognize, often biases can be reconstructed or reinterpreted as legitimate debates about values.

KIMBERLY NEVALA: But you also make a point about shifting accountability from systems to individuals. And again, I think this goes back - in my mind at least, may not have been your intention - to this larger question of innovation. Which is, what is it that we are actually innovating towards and innovating against when we think about these systems? But can you talk a little bit about that?

Why or how, if we are not mindful about this, essentially, you said it, not only it does erode accountability, and at the same time, it sort of shifts it to individuals. Which would seem, on the surface, almost like contrary or contradictory statements. But I think this is something that organizations, and then, at large within society, when we're thinking about policy and where to apply these and how to think about these decisions, we probably should be attending to a bit more often.

STUART MILLS: Ask me to clarify if this doesn't come across clearly enough, but I think a distinction that is important to make, and we don't make it often enough, is that most things that we think are decisions are actually just processes. Whether or not we launch a new product, that's the decision. Whether or not that product is going to be launched with five colors or six colors, that's more of a process. That's determined by our manufacturing and our design capabilities.

And so when we apply technology, when we bring algorithms into organizations or AI into organizations, it's often just to automate processes. And this sounds perhaps trivial, but an algorithm is just an automated process. It is just a do this, then do this, then do this. But because we often confuse processes and decisions, we can often apply technology in very, very poor ways. Because our processes, we think of them as decisions. Many of our decisions, we also think of them as processes.

And so you end up putting an algorithm in charge of deciding should we launch a product? This is a silly example, but you would never put an algorithm in charge of whether your organization should launch a new product. Actually, some places do, but that's beside the point. Some Amazon products are algorithmically generated, but that's neither here nor there.

But if we confuse what is a process with what is a decision, it's very easy for us to put algorithms in charge of decisions. And at that point, we shift who has to demonstrate intuition in an organization, who has to demonstrate discretion, and we shift who is ultimately accountable.

To give you a very, maybe it's a very similar example, maybe it's very different, before we started recording, we had a little chat about bureaucracy and filling out forms and things like that. The more and more forms that you have to fill out to access, say, Social Security or some other government benefit, the more it becomes important that a human being can demonstrate discretion.
So in the UK, for instance, at least 15 years ago, you could go into a post office and there'd be a human being there. You could get the various forms you needed to interact with the state. And the human being would have training to be able to help you navigate those forms. Whether or not they should, they had the ability to demonstrate discretion. And it kind of made the system work, because you can file an awful lot of forms if you've got a human in the end who's helping citizens navigate all those forms. Fast-forward 15 years. Lots of post offices have closed down. Lots of the state has moved online. Those forms still exist, but the human being who is demonstrating discretion to help people navigate it, they've disappeared.

And so what on paper should be creating more efficient systems, because we've automated everything and it's all online, you can access it at home and so on, actually, at least for some people, makes the system so much harder. Because we've automated not the process - the process of filling out the forms, that can be automated. But what we've automated is the person who can demonstrate discretion and help a person navigate the system. The person who can make the decision to say, I will step in and help. We've gotten rid of that point.

I worry that was a bit incoherent, but I also think there's a degree of intuition to what I'm saying. Most of us have interacted with bad bureaucracies.

KIMBERLY NEVALA: Well, and this jumps me forward to one of the most, I think enduring, probably, stories of technology in general - and not just limited to AI, and it's an economic story as well - is always around productivity. Productivity is a measure of, at some level, maybe its economic health or growth. You're the economist, you're going to correct me on that one. But it's also this pervasive discussion about why AI?

Why AI, and all different flavors and forms of AI. But particularly today, why generative AI, why agentic AI, and this issue. And so how do you then think - or does this relate at all to this - how do you think about the AI productivity narrative? And what does that not necessarily get wrong, but what nuance, or are there nuances, that we are losing sight of even in that conversation as well? When that is the driver of decision-making about this tech.

STUART MILLS: I mean, I should forewarn you, I could talk for hours on technology and productivity, because there's so much ground to cover. I think most people recognize, at a certain point, that all the discussion about AI right now boils down to this desire for greater productivity.

And the background to all of this is, at least since the Global Financial Crisis in 2008, Western economies have seen declining productivity growth. Some countries have performed much better than others. The US has performed much better than my country, for instance. But productivity growth has gone down.

And productivity growth broadly, if we want just one metric of prosperity, that is it. It measures how much stuff we produce. And if we, as a society, can produce more-- setting aside questions of distribution and inequality-- the more we produce, the more wealth we, as a society, have. And obviously, people want more stuff. People want more security, which comes from having more stuff. So productivity is really, really important.

But the current discussion is against the backdrop where productivity growth has been declining so there's this mad dash to find some way of boosting productivity. The question around productivity growth, at least in Western economies, increasingly has to focus on knowledge work, because Western economies in the '70s and '80s got rid of a lot of their manufacturing. A lot of it went overseas, globalization and so on.

And Western companies said, we will focus on training people who know a lot of stuff, who have a lot of expertise, designers, programmers, managerial talent, things like that. And we'll transition to a knowledge economy. Everyone will work in an office, and we'll all work on computers, and the actual manufacturing will be done somewhere in the Global South. And integrated supply chains, that's how we'll run our economies.

So if you want to create productivity growth in a modern, post-industrial economy, you've got to make knowledge workers more productive. And that's how we arrive at AI. It's a very simple story, but I think it's broadly correct. You don't see these developments in AI in an economy that is not quite as knowledge intensive as, say, the United States or the European economies.

The final part of this story is, well, does AI actually make us more productive? And the honest answer is we don't know. But for every paper that says it's making us more productive, I can show you a paper that says it's not. And we can debate endlessly how we're measuring productivity. For instance, there are several popular papers looking at generative AI and productivity growth. Often what those papers measure are quite artificial measures of productivity. It's getting people to produce widgets. But in an economy that doesn't produce widgets, does that mean AI is actually making them more productive? It might be getting people to write more reports or write more code, but actually, the value of those reports is not in the quantity. It's in the quality. And if we're not evaluating the quality, can we really tell a story about productivity?

So the long and the short of it is we really don't the productivity effects. And we won't know for a long, long time. And some people might think I'm crazy when I say this, but I'm part of a group of people who think that we don't actually know the productivity effects of computers. There's so many nuances to how computers work and how computers convey value, that there are still ways we can debate whether the computers made us radically more productive. And it's the same story with AI.

So we have to be cautious to not get caught up in the hype. And I'm not beyond believing AI can make us more productive. But I am certainly beyond believing it will be a noticeable step change in productivity. To sign off, so you can ask me more questions rather than me just keep rambling, it's a useful exercise, I think, to look at the productivity effects of previous epochal technologies.

Electricity has multiple-thousand-fold increases in productivity. The steam engine, similarly, thousand-fold increase in productivity. Electric light is one of the most important inventions in human history. It extends the workday tremendously. It allows us to do things in the evening, where our ancestors never could. That's transformative technology. Look at the computer. Someone like a Robert Gordon would argue that there has been a productivity boost from the computer, but it was a one-time step change, and it was a step change of tens of percent, if that.

And we live in the information era. We think computers are radically transformative. And in social ways, they have been. But in economic productivity ways, much more modest than the electricity revolution or the steam revolution or even electric light. And so when I think about how productivity enhancing could AI be, I have to put it in the context of history and say, well, it might have some positive effect. But it's not going to transform society in the way electricity did.

So we've got to be really cautious and avoid the hype when people say it's going to change everything. It will have an impact, probably a modest economic impact. Certainly, we're seeing a sizable social impact. But it has to be put in the historical context to really understand what a big impact actually is.

KIMBERLY NEVALA: And you had said something to me, and I wrote it down when we were chatting briefly before, and you said we have to be really cognizant of what is productive versus what actually drives productivity. And if we're asking those questions and thinking about that critically, we might organize ourselves, we might think about how we organize ourselves, as humans differently. And, as a result, build different AI systems to support that. Did I get that correct?

STUART MILLS: Well, that's more or less my perspective. I mean, I'll give an example. And it's my favorite example, because it's a controversial example. In about 2023, a whole bunch of investment banks announced that they were going to start using AI systems to automate their employee annual review sort of thing. I'm sure we've all been through them. I'm sure most people have varying views of whether these meetings actually add to the productivity of the business. Are they valuable? I'm going to make the bold assertion that they are actually valuable. Maybe I don't actually believe that, but for the sake of argument, let's say they are actually valuable.

Let's ask the question, where is the value created in an annual employee review, where you meet your boss and you have a talk about what's gone well and what's not? The value comes from the interpersonal interaction between you and your boss, where you can say, this is what's gone well. This is what hasn't gone well. And your boss can say this is what I think you need to improve on. And you can both work together to create a plan for you to succeed. And let's assume your success ultimately leads to the productivity of the business. It makes you a better worker. The value comes from that interpersonal interaction.

What were these investment banks going to do with AI? It was going to be the employer, sits down and types their thoughts and feelings into an AI system. The AI system would then summarize it for the boss, and the boss would be able to tick off, yes, I've reviewed this employee. Totally automating the value component of the interaction and thus destroying the value whilst maintaining the activity. That's not going to enhance productivity in the long run. If anything, if we assume that these interactions do genuinely add value, the AI would be destroying value in this instance.

KIMBERLY NEVALA: One of the other things that, in that example, even, highlights is this idea that that process sometimes can feel, to use the term perhaps incorrectly, sludgy. It can feel hard. It has friction involved in it. And today, we're so focused on making things frictionless, making them fast, making them easy. Or just making them unseen. As you said, to still do them, but pretend that they're not happening or that they're under the level of that-- that we're actually eliminating maybe helpful friction. And the friction between that conversation and your boss is part of the point, again, assuming that that's done correctly.

The friction-- I had a conversation a while back with Henrik Skaug Sætra, who I know you know as well, about democracy. And he said, yeah, it's a messy, sticky, hard process, and that is the point. That is the work to be done. And so I wonder if, in total as well, this ties back in some way to that discussion as well, which is frictionless is maybe not always the objective we should be looking for, either.

STUART MILLS: We live in a very bizarre moment, where on the one hand, a tremendous amount of stuff in everyday life is frictionless because of technology. But on the other hand, rather than it being we can make choices in a frictionless way, it's often we can pursue one path or one process in a frictionless way. And if we wanted to do something else, that would be overwhelmingly difficult to do.

Obviously, on an individual level-- Silly, silly example. I don't use Spotify. I still use an iPod. I've got it behind my desk. It's not that difficult to do. I grew up in the era of using MP3 players. I know how to do it, but it's obviously much more difficult to do, particularly if you've never used this antiquated technology before, than just using Spotify. So the added friction discourages you from consuming music in one way, and consuming music in this other, more frictionless way.

Most people, I think, when we talk about - or at least from my experience - most people, when we talk about friction or sludge or how this stuff is becoming problematic, will often point to how, increasingly, it seems interacting with government is harder, and increasingly how businesses make it difficult to interact with them in ways which businesses don't necessarily want to.

The obvious famous example is canceling a subscription. Most services, most businesses will make it the easiest thing in the world to subscribe to them and one of the most psychologically damaging things that you can imagine to try and unsubscribe. And we have done research about this before, and I'm not exaggerating when I say it can be two, three times more difficult to unsubscribe to something than to subscribe to it. And that's on the good side. That's the businesses that are doing things well.

So we live in this bizarre moment, where things are both simultaneously lacking in friction. But also, if you want to take your own path, if you want to go against the frictionless path, you're not met with a good or a positive amount of friction. Sort of, are you sure you want to do that, sort of friction? But you're met with an overwhelming, no, we are going to do everything in our power to make this as difficult as possible kind of friction.

And that creates, I think for most people, certainly for consumers we've spoken to, it creates a really bizarre mindset where you feel disempowered. But, also, you're able to interact with the world. I'm struggling to find the language, but I'm trying to articulate there's a strange paradox when we think about friction right now. That's my inelegant conclusion.

KIMBERLY NEVALA: Well, like yourself, I can talk and ask questions about these things for hours and hours and hours. I endeavor not to, but there is a reason that our original aspiration to be 20-minute podcast episodes never came to be. Blew that out of the water, episode one.

So two last questions, just to do that. And we will link to a lot of your work, which I think is thought-provoking and quite important at this moment in time. Is there, when it comes to making or thinking about policy or decisions in practice broadly that you would point to for organizations and individuals that should be top of mind? Or a question we should be wrestling with as this moves forward?

STUART MILLS: I had this line, which I tried to make popular, but I'm not much of an influencer. Radical technologies should lead to radical change. It sounds quite trivial. It sounds obvious. But I've spoken to dozens of business leaders in the past few years - or business people. And the number of them who are really, really excited about adopting AI, but who are totally hesitant to actually fundamentally changing the structure of their organization and the way their organization works. Most leaders, most executives do not want to change the sight of decision-making. They don't want to shift power around in terms of who can make decisions and who has authority, who needs to be checked and who doesn't.

And setting aside AI for a second, to go back to what I said about electricity, you don't get the massive productivity gains from electricity if you're not willing to electrify the factory and change the layout of the factory to maximize that electrical infrastructure. Similarly, if we assume that AI can radically boost productivity, it's only going to happen through radically restructuring how organizations work.

And at least in my opinion, there seems to be a tremendous hesitancy from executives to really change the structure of their organizations. Or to give up or change who has power within organizations. Hence, why I'm perhaps skeptical of the ultimate productivity impact of AI.

KIMBERLY NEVALA: Absolutely. And at this moment in time, I will say I'm worried that those who are willing to make radical transformations are not at the level of - within societal organizations or even in our standing corporations and things - that are outside of a lot of the political and social environments that might ensure that any such radical transformation is actually positive and is a net good for the most people. But I think that is a great line to leave on.

And although I do have to ask, because I know you are a voracious reader and put up great things, what have you read of late? Or it actually doesn't even have to be of late, that you think has most shifted your perspective or influences your thinking? That you would, if I said you had to make one recommendation for folks to help them think differently or understand better how things like behavioral economics, behavioral science are intersecting, and impacting how we think about and engage with tech, what would that be?

STUART MILLS: I mean, I should say, if people want to see what I read, I post it all on my website. But the books that come to mind, if people are interested in behavior and technology, and how they interact, there's a fantastic book from a couple years ago called If Then by Jill Lepore, which is a history of a very strange company called Simulmatics or something like that. This was a company set up in the late '50s that collected a whole bunch of data on American voters to try and predict voter preferences and influence their voting behavior during the 1960 presidential election. This is decades before Cambridge Analytica and all the stuff on social media.

And I just read this whole book. I thought, firstly, why have I never heard of this? Everyone should be talking about this in my world. And secondly, there's so much crossover between what happened with Simulmatics and what's happened in behavioral science, at least in my part of behavioral science, in the past 10 years or so. So it's a brilliant book. I'd recommend people read it. Jill Lepore is a New Yorker writer as well, so she's a wonderful writer.

The second book I'd recommend came out in May of this year by a guy called Benjamin Recht. It's called The Irrational Decision. And it's basically a history of rationality, and how the 20th century invented the idea of human rationality. And Recht's whole point is that rationality has distorted our understanding of what it means to make good decisions. It's distorted our understanding of how we should use technology. And this obsession about rationality and chasing rationality has created all sorts of social perturbations that continue to play out and cause all kinds of chaos in society. It's a brilliant book. I really recommend it.

The last book, because you did say what had been influencing my thinking, I wanted to recommend quite an old book. Now, I mentioned Richard Levins and Richard Lewontin earlier. The book is called The Dialectical Biologist. It's a very strange book. It's a strange book for an economist to read, and I read it in my departmental meeting a few months ago. And all my colleagues were looking at me like, why is he reading a book on biology?

But it's got a whole bunch of stuff about science and ideology. It's also got a whole bunch of stuff about moving beyond individual analysis of problems, and moving beyond seeing the world as an individual experiences it, and thinking about how individuals and the environment interact and influence one another. I can't quite explain why I think it's so interesting and so influential, but I'd encourage people to read it, and maybe they'll get some of the magic out of it that I got as well.

KIMBERLY NEVALA: Awesome. Well, on that note, I will resist the urge to continue asking you questions for the next several hours and maybe leave you with an invite to come back and we can talk some more about this in the future. But thank you so much for your time and insights and the book recommendations; which I will be adding to my running list immediately hereafter.

STUART MILLS: Thank you so much for having me.

KIMBERLY NEVALA: Awesome. So to continue learning from thinkers, doers, and advocates such as Stuart, you can find us wherever you listen to podcasts and also on YouTube.

ANNOUNCER: This has been a SAS podcast.