#55 - Why modeling 'I don’t know' is AI's most urgent problem - Ruchir Puri

AI systems almost always have an answer, even when they should say, “I don’t know.”

In this episode of The Deep View Conversations, senior reporter Sabrina Ortiz speaks with Ruchir Puri, chief scientist at IBM Research, about why uncertainty modeling may be AI’s most urgent technical challenge.

Puri explains why today’s models struggle to recognize the limits of their own knowledge, how that failure contributes to hallucinations, and what researchers must solve before AI can become more reliable. He also explores the need for self-improving models, the enormous energy gap between artificial and human intelligence, and why the future of AI depends on doing more with less compute.

The conversation also covers:

• Why Puri predicted in 2020 that AI would transform software development
• How big data, GPUs, and transformer architectures created the current AI boom
• Why intelligence involves more than IQ
• The roles of emotional and relationship intelligence
• Why language models cannot capture the full complexity of the physical world
• How AI could help redesign software, quantum computing, and chip development
• Why Puri prefers "artificial useful intelligence" over AGI

Rather than chasing abstract definitions of general intelligence, Puri argues that the industry should focus on building AI that is useful, efficient, adaptable, and honest about what it does not know.

Subscribe to Deep View Conversations for interviews with the leaders shaping the future of AI, business, and technology: tdv.transitor.fm 

And don't forget to sign up for The Deep View daily newsletter. We don’t just cover AI, we decode it. In a world flooded with hype, we deliver sharp, no-nonsense insights to keep you ahead of the curve and help you put AI to work every day: subscribe.thedeepview.com

Creators and Guests

Host

Sabrina Ortiz

Senior Reporter at The Deep View

What is The Deep View: Conversations?

From frontier labs and enterprise platforms to emerging startups reshaping entire industries, The Deep View: Conversations podcast interviews the brightest minds and the most influential leaders in AI.

Sabrina Ortiz: Thank you so much for joining us today. I would love to kick off the podcast by having you tell me a bit about who you are, what you do, and how you got involved in the AI world and the AI space we’re in now.

Ruchir Puri: I’m chief scientist of IBM Research, and I’m in a very broad role. But my journey into this started very much with my research topic way back in the early ’90s. My background is in algorithms, which are the foundations of almost everything that is happening around us, but even more importantly, AI. In particular, my journey in AI started in 2011, way before ChatGPT was even a word, where I was accelerating workloads on GPUs before NVIDIA was on its hyper-trajectory as well. Then, over time, I took the role as CTO of Watson for four years, and then I came back into research in a broader role as chief scientist. I have been involved in AI for more than 15 years now.

Sabrina Ortiz: It’s incredible. I was looking up your LinkedIn, and very few people have LinkedIns that leave me that much in shock, like, wow, all this experience. You started at IBM, what, 30 years ago?

Ruchir Puri: Thirty-one years, to be precise.

Sabrina Ortiz: That is incredible. And in the first role, again, I was super surprised to see that it was autonomous algorithms. I feel like that’s a word we would hear in every press release, press conference, and keynote today, but 31 years ago, that’s what your role was. Can you tell me a bit about what it looked like then compared to what that same kind of role would look like now?

Ruchir Puri: So, as I said, I’ve been in the business of, if I may say, automation for three and a half decades. Before any of the world of AI, we were deep in the winters of AI at that time. It was not something that you would utter in an interview; it’s something you hid from everyone else that you ever did. And back then, I think we went through a couple of phases. It’s not that AI itself—the word was coined in that famous meeting at Dartmouth in New Hampshire, where 16 people, and a couple of IBMers, were there as well and coined the words “artificial intelligence.” But the underlying foundations of it have been algorithms, like what is known as gradient descent. If I may describe that algorithm in its simplest form, it’s like you are trying to traverse a complex, hilly terrain, and you’re trying to get to the lowest point in a valley, but you can’t see. You can’t see around, because the lowest point in the valley is somewhere. You are standing on a small hill, and it may be beyond a big mountain, down the slope somewhere. So you say, okay, what’s the best path you follow? You look around and say, what’s the steepest path? Then you go there, and you keep following the steepest path. At some point, you say, I think I’m stuck in a kind of a local valley. Then you randomly, somehow, hop on a drone and get dropped somewhere else. That’s the best way to describe that algorithm. That is kind of the foundation of—I’m really oversimplifying it—but that’s the foundation of how we train these models with massively large data. What has happened over the last three decades is that, at that time, we had neural networks, yes, but we didn’t have big data. The internet had not exploded yet, so there was no data available. You had more analytics—very precise algorithms for predicting things. Neither did we have big compute. So what has happened since then is that three things have come together. There was the internet explosion, which resulted in a massive amount of data. If you have a massive amount of data, then you are kind of handicapped if you don’t have enough compute to process it. Some researchers figured out in a lab at Stanford that these GPUs that work on graphics work very well on matrix computations. So now I can take this big data and marry it with big compute. There were some advancements that happened in AI techniques as well, in particular transformer architecture, which is now the foundation of the entire large language model area. These three things, when married together, result in this phenomenal breakthrough. None of that existed back then. So that’s the main difference. I would say big data, number one; big compute, number two; and advancements in algorithmic techniques as well. That’s really the supernova you’re looking at right now.

Sabrina Ortiz: I’m curious when you yourself felt a shift. You were like, this actually has potential to be so much more than it is now.

Ruchir Puri: I would say 2020.

Sabrina Ortiz: Okay.

Ruchir Puri: This was pre-ChatGPT.

Sabrina Ortiz: Right.

Ruchir Puri: Many people in IBM certainly know it, and outside know it as well. I had predicted this back in 2020—not just a shift. I said, I think AI is going to change coding. It’s going to fundamentally change software. And I said, since every company that we know of today is a technology company and a software company—the world around us is software. We don’t feel it as software because we consume it as services, but underlying it is software. The bank is a software company. Every logistics company is a software company. Software is everywhere. If we could do something more fundamental with software, we’ll change the world. Back in 2020, I was working on something called AI for Code—AI is going to change coding. Why, you may ask, is that the case? Because with software, we are blessed with the fact that there’s something called verification. Software is not kind of right. It’s either right or wrong.

Sabrina Ortiz: Yeah.

Ruchir Puri: Now, language is kind of different. Language has a lot of gray zones in between interpretations. Your interpretation may be different than mine, but software is like—you produce something, and you can verify it.

Sabrina Ortiz: Right.

Ruchir Puri: And that verification is golden, because then you can say, these were the errors I got. You produce something, you run it, you verify it, and say, these are the errors I got. You send it back and say, correct these errors. There’s a closed iteration loop that can happen, which is kind of a blessing in software. So, you may say, where was my aha moment? I was playing with these models at that time, which were, I would almost say, two to three orders of magnitude smaller than where we are today. It was kind of the moment for me when I thought, this is going to change the game. I remember grabbing lots of, if I may say, authorities in the area and saying, this is going to change the game. And they said, no, I think you are hallucinating. I think what happens is that, if you are in a particular area for too long, then you can’t see the disruption coming easily because you know too many of the ills of the area. So I took it upon myself to gather a massive amount of data that we now call CodeNet, which has actually gone into AlphaCode from DeepMind and many others as well. To me, I remember telling people who were responsible for a lot of software in our business, this is going to change the game. I think they partially saw it and said, you said this is going to take my job. And I said, not exactly. That’s not what I meant, practically. But I really think it’s been six years for me in which it was clear that we are in for a massive transformation.

Sabrina Ortiz: I would ask you about all of the positives and the potential use cases and the potential disruption, but I think at this point we’ve all seen it. We’ve all either interacted with a chatbot to a smaller degree or used it or implemented it in a workflow where we’ve seen the positive impacts. We know that they’re here, and we know that disruption is here. We see positive cases where, like today, when we found out about the quantum example, something that could not have been done before is being done now. We’ve seen that, but we’ve also seen the negative use cases, where people’s jobs are being replaced, people are being laid off, and all of that. So instead of focusing on the impact, I would love to pick your brain a bit about the actual algorithms, since that’s your bread and butter. Do you think there’s a world, or any research that it’s possible to do—and how long do you think that would even take—to start eliminating some of those algorithmic hurdles that we have right now? For example, hallucinations, being overly confident, all of those things. I’m sure you’ve seen them from the beginning, and now they’re still an issue. When do you think we’ll see the next step there?

Ruchir Puri: I think there are, I would say, two fundamental issues, or two fundamental aspects, that are still hot topics of research. Let me step back and say: knowing what you know is retrieval. It’s in your memory. You retrieve it—good. That’s maybe part of intelligence. Knowing what you don’t know is intelligence.

Sabrina Ortiz: True.

Ruchir Puri: Like, I don’t know this. There aren’t many people who acknowledge, “I don’t know this,” but internally everybody knows what they don’t know. Whether they say it or not is a different thing. I think it’s kind of hard to lie to yourself because you know what you don’t know. In my mind, these models are still learning to model what, in technical or algorithmic terms, is called uncertainty. When was the last time you or anybody else had an experience with a chatbot or a large language model saying, “I don’t know”? They always tell you an answer. Then, if you push them hard, they’ll say, “Oh, I’m sorry. Actually, I was incorrect.” I think it has happened with each and every one of us. This, to me, is a fundamental aspect of research that we are doing in IBM Research, and that lots of institutes in academia and other industry frontier labs are doing as well, which we call the ability to model your own uncertainty and to reason with that uncertainty as well. So that’s one aspect. I think the other one is an aspect that’s fundamental to humans: self-improvement. You start from a certain point, you continuously learn from things, and you continue to self-improve. Large language models are still at a point today where they’re given a certain intelligence, and you get what you get, and you kind of don’t get upset. They are at a certain intelligence, and you wait until the next release of the model, chatbot, or agent for you to get the next level of intelligence. Humans are not like that. You see Ruchir today, and Ruchir may have a different version nine months from now. It doesn’t work like that. I may be different and continuously improving, including from this discussion we are having. My responses next time around will be informed by the journey I’m going through all the time. That is called self-evolving on a continuous basis. That is another foundational aspect that we are currently grappling with and dealing with in research. Obviously, a tremendous amount of progress has been made to date on making AI useful, but still a large gap remains in bridging the gap to, I would almost say, making artificial as good as real. The third aspect, I would say, is that everything in the world is about efficiency. I’ll elaborate on that a little bit. Given the energy of the sun, everything is possible in this world. Given the energy of the universe, everything is possible. But now, if you look at our own intelligence—whenever there is something artificial, there must be something real. That’s why there’s “artificial.” Artificial doesn’t exist in the absence of real. That real is sitting between these two ears. It sits in 1,200 cubic centimeters right here. It consumes 20 watts—the power of an LED bulb on your phone—and runs on sandwiches.

Sabrina Ortiz: Yeah.

Ruchir Puri: That artificial—a single NVIDIA Blackwell B200 GPU—consumes 1,200 watts, 60 times more than your brain. And you need hundreds to thousands of them to do even partially what a brain can do. That is the gap you are looking at between artificial and real. That’s a journey we are on: to make intelligence more and more efficient. Stanford had a very nice paper recently called “Intelligence Per Watt.” I think that’s the third leg. We’ve got to make it more efficient. I just take objection with, you know, “I need the power of the city of Columbus, Ohio, to run.” This is just not heading the right way. Now, we are brute-forcing it. That doesn’t mean to say that’s where we will end up.

Sabrina Ortiz: Yes.

Ruchir Puri: So those are the three major things. I would say those are the gaps remaining where we need to get to. Energy efficiency is a fundamental one, where you can address it with two aspects: you have infinite energy available, like fusion. You address it with the energy aspect: “Hey, I make energy free.” You can see with all the geopolitics going on, energy is not free, for sure. Or you can make intelligence more efficient, which is an aspect I think collectively many of us are working on as well.

Sabrina Ortiz: I would love to double-click on the intelligence aspect of it all, because what these models are doing—and I’m definitely oversimplifying, and I would love to hear your take because you could explain it much better—but they’re essentially taking all this data, analyzing it, and then making predictions based on the patterns. Well, as you said, that’s part of what we do. We recall our prior experiences, our education, and our learnings, and then use that to make a decision. There are also a lot more factors than just recalling data. We have our own experiences, our biases, our emotional process, all of that. So that’s what I would say is more akin to what I would call intelligence, or thinking and reasoning. Will there ever really be a way to mimic that artificially? To your point, to mimic what’s in here and put it in a chatbot or an artificial algorithm, when it can’t do those things?

Ruchir Puri: So I think you’re getting at something very fundamental. I’ve said this for—I remember interviews with many media sources 10 years back on, where will intelligence be? And I said, what is intelligence? In my mind, intelligence comprises three things. One of them is very well known, and everybody talks about it: IQ, intelligence quotient. I think AI will nail IQ—literally will nail IQ. The second aspect is EQ, emotional quotient. I think we would all agree—let’s consider successful CEOs, or CEOs of most corporations. We would agree all of them are highly intelligent people, but all of them have amazing EQ, at least all the successful ones. But they may not have the highest IQ. I don’t mean to say they are bad—not at all. I mean to say they may not be the geniuses of the world. They’re amazing people for doing strategy and really dealing with large organizations and people. So it needs a mix of these multiple quotients. The third quotient that doesn’t get talked about as much is what I call RQ. RQ is relationship quotient. It’s kind of related to EQ a little bit, but not exactly: how good you are at forming and growing relationships. Again, I think it benefits from the other two, but it’s a separate factor in itself. The combination of IQ, EQ, and RQ gives rise to intelligence. I think AI will nail IQ. It’s still far away from EQ, I would say, because EQ is more like, you can see somebody is distressed, and you say, “Give me a hug right now.”

Sabrina Ortiz: It’s also subjective, right?

Ruchir Puri: It is.

Sabrina Ortiz: Everybody kind of handles it a bit differently.

Ruchir Puri: Yeah, everybody handles it a bit differently. It’s very contextual. We haven’t matured yet to a point where a machine giving us a hug will calm us down. Not yet. Maybe someday we will. But I think just the power of human touch and human emotions is something that is innate in us as humans. Of course, the machines may become amazing at automation. I think they’ll get infused into every aspect of our daily lives, but these aspects of EQ and RQ still remain a goal for us. That, to me, is part of intelligence that we need to continue to work on. I do think we may be able to predict emotions, by the way. It’s certainly possible that I can predict somebody is distressed. But to be able to deal with those emotions, calm them down, and give them a hug—I think humanity is a lot more than can be captured in tokens. A lot more than that.

Sabrina Ortiz: Yeah. I almost feel like there’s kind of a field around each one of us where you don’t have to say anything. No tokens exchanged. I didn’t say anything, but the other person knows what I’m feeling.

Ruchir Puri: There is no such thing as of yet. And then there are people like Yann LeCun and others who are talking about world models. So I think we will all agree token-based intelligence gets us to a certain point, which is language. I think language has massive implications, and it has done amazing things, but the real world around us is a lot more than can be captured in tokens. That’s the part I think we are still missing, in terms of whether it is world models—how do you model the world around you?—or other technologies. There are still gaps and unknowns that remain.

Sabrina Ortiz: We’ve talked about a lot of challenges, from actual intelligence ones to the emotional aspect. Which one do you think is the most important we tackle now, and which one are you the most hopeful we’ll get tackled soon?

Ruchir Puri: I think the one we must tackle now is one of the aspects that I mentioned among the three: uncertainty modeling.

Sabrina Ortiz: Yes.

Ruchir Puri: If these technologies are to become useful, they need to reduce, I’ll call it, more of their lying—also known as hallucinations. If you hallucinate a whole lot, I think you’re lying.

Sabrina Ortiz: But, I mean, lying is something where you know that you knew it, and you’re still saying it.

Ruchir Puri: They don’t know it, actually.

Sabrina Ortiz: It’s almost scarier, right? Like almost having a pathological liar around you all the time. They’re not even aware they’re doing it.

Ruchir Puri: So I think we need to address that particular aspect of knowing what you don’t know. That is urgent. That is here. And, again, as I said, it’s a very, very popular topic at all leading conferences in the AI area. You go to NeurIPS, you go to ICML, you go to ICLR, and many other conferences as well. The second one we need to address—and this starts leading us to efficiency—is self-evolving, learning from your experience. Then you will not need the power of a gigawatt farm to be able to feed it. We don’t need a gigawatt of power attached to us anywhere. We are running on sandwiches on a continuous basis, and we are improving on a continuous basis as well. I think that is the second one. The third one, I think, relates more to hardware technologies and computer architectures: starting to work on technologies in semiconductor chips, computer hardware, and accelerators. How do you make these foundational components of intelligence, or artificial intelligence, a lot more power efficient as well? In that order, I would say: address uncertainty now. Second, as you start to tackle it and reduce hallucinations, you start to improve on self-learning and adaptation. The third one is, longer term, continuing to close the gap between real and artificial.

Sabrina Ortiz: Despite everything we just said, I still get the sense—from your commitment to really working in this for so long—that you are still invested in the potential AI can have, the potential AI has now, and that you’re still somewhat bullish on AI. Why is that? What makes you still confident despite all these challenges?

Ruchir Puri: So I’m a huge fan of a word called—or that sort of icon called—AUI. There are a lot of people talking about AGI. I kind of roll my eyes a little bit, because we don’t even know—as we talked about, intelligence is multiple things. So this “general intelligence” kind of bothers me a little bit. I like the word AUI, which is artificial useful intelligence. Can somebody make it useful for me? That’s what I care about: it’s useful. In that vein, I am not just optimistic; I’m very bullish on AI technology being useful in day-to-day life. It can make somebody who doesn’t speak a language converse in that language. It can make my customer experience much better. It can make software tasks that many people find very tedious more efficient. Those are all amazingly useful things that are there in our day-to-day lives. Whether we have achieved AGI or not, leave that aside.

Sabrina Ortiz: Totally.

Ruchir Puri: And that’s why I am extremely excited about the power of AI technologies to be useful today. I think Arvind talked about it earlier in the morning as well, that there are two ways of approaching it. One is what we will term “plus AI,” which is: I’m given a task, and I’ll say, how can I do that task a little bit better? Or I can infuse AI here, I can infuse AI there, and I’m better. That, to me, is incremental. The second one is what Arvind described as “AI plus,” which is: if I knew AI technology existed, would I do this task completely differently? Literally reimagine this task as well. That, to me, is the true promise of AI technology. I think lots of companies today are invested in making “plus AI,” which, I would say, yes, will get you some productivity improvements—and impressive ones too, many times. But if you want to really change the game and get really massive jumps in productivity and usefulness of the tasks, then you need to shift to reimagining the business processes and the tasks themselves. Again, we are doing this at IBM. We are doing this in research for things like quantum itself as well. How do we discover quantum error-correction codes? Fault tolerance in quantum is, I would say, a major, major milestone that we have said will occur in 2029. How can AI help us get to better error-correction codes? How can AI help us compile the quantum codes much better? How can AI help us literally reimagine how we calibrate these systems as well? How can AI help us do chip designs much better, in terms of just reimagining the whole process—not just, AI can help me do one algorithm better. If I were to do this all from scratch, what would it look like? Those are the things we are deeply invested in, to make these technologies not just useful, but really redefine the era of how these tasks are done.

Sabrina Ortiz: Well, on that positive note, I think it’s a great place to wrap up our conversation. Thank you so much for your time. This was really insightful.

Ruchir Puri: Thank you. I appreciate it.