Subscribe
Share
Share
Embed
Fixing the Future from IEEE Spectrum magazine is a biweekly look at the cultural, business, and environmental consequences of technological solutions to hard problems like sustainability, climate change, and the ethics and scientific challenges posed by AI. IEEE Spectrum is the flagship magazine of IEEE, the world’s largest professional organization devoted to engineering and the applied sciences.
Hi. I'm Dina Genkina for IEEE Spectrum's fixing the future. Before we start, I wanna tell you that you can get the latest coverage from some of Spectrum's most important beats, including AI, climate change, and robotics by signing up for one of our free newsletters. Just go to spectrum.ieee.org backslash newsletters to subscribe. And today, our guest on the show is Suraj Bramhavar.
Dina Genkina:Recently, Bramhavar left his job as a cofounder and CTO of Sync Computing to start a new chapter. The UK government has just founded the Advanced Research Invention Agency or ARIA, modeled after the US's own DARPA funding agency. Bramhavar is heading up ARIA's first program, which officially launched on March 12th this year. Bramhavar's program aims to develop new technology to make AI computation 1,000 times more cost efficient than it is today. Suraj, welcome to the show.
Suraj Bramhavar::Thanks for having me.
Dina Genkina:So your program wants to reduce AI training costs by a factor of a 1000, which is pretty ambitious. Why did you choose to focus on on this problem?
Suraj Bramhavar::So there's a couple of reasons why. The the the first one is is economical. I mean, AI is is basically the become the primary economic driver of the entire computing industry. A £100,000,000 now. And AI is really £100,000,000 now.
Suraj Bramhavar::And AI is really unique in the sense that the capabilities grow with more computing power thrown at the problem. So there's kind of no sign of of that those costs coming down anytime in the future. And so this has a number of knock on effects. If I'm a world class AI researcher, I I basically have to choose whether I go work for a very large tech company that has the compute resources available for me to do my work, or go raise £100,000,000 from some investor to be able to do cutting edge research. And this has a variety of effects.
Suraj Bramhavar::It it it dictates, first off, who gets to do the work, and also what types of problems get addressed. So that's the economic problem. And then separately, there's a technological one, which is that all of this stuff that we call AI is built upon a very very narrow set of algorithms, and an even narrower set of hardware. And this has scaled phenomenally well. And we can probably continue to scale along kind of the known trajectories that we have.
Suraj Bramhavar::But it's starting to show signs of strain. Like I like I just mentioned, there's an economic strain, there's an energy cost to all this. There's, logistical supply chain constraints. And we're seeing this now with kind of the the, GPU crunch that you read about in the news. And in some ways, the the the strength of the existing paradigm has kind of forced us to overlook a lot of possible alternative mechanisms that we could use to do kind of perform similar computations.
Suraj Bramhavar::And this program is designed to kind of shine a light on those alternatives.
Dina Genkina:Yeah. Cool. So you seem to think that there's potential for pretty, impactful alternatives that are orders of magnitude better than what we have. So maybe we can dive into some some specific ideas of of what those are. And you talk about in your thesis that you wrote up for the start of this program, you talk about natural computing systems.
Dina Genkina:So computing systems that take some inspiration from nature. So can you explain a little bit what you mean by that, and what are some of the examples of that are?
Suraj Bramhavar::Yeah. So when I say natural based or nature based computing, what I really mean is any computing system that either takes inspiration from nature to perform the computation or utilizes physics in a new and exciting way to perform computation. So you can think about, kind of people have heard about neuromorphic computing. Neuromorphic computing fits into this category. Right?
Suraj Bramhavar::It is it takes inspiration from nature and and usually performs a computation, in in most cases, using digital logic. But it that represents a really small slice of the overall breadth of technologies that incorporate nature. And part of what we wanna do is is highlight some of those other possible technologies. So what do I mean when I say nature based computing? I think in our we have a solicitation call out right now, which calls out a few things that we're interested in.
Suraj Bramhavar::Things like new types of in memory, computing architectures, rethinking AI models from an energy context. And we also call out a couple technologies that are pivotal for the overall system to function, but are not necessarily so eye catching. Like, like how you interconnect chips together. And how you simulate a large scale system of any novel technology outside of the digital landscape. I think these are critical pieces to realizing, the overall program goals.
Suraj Bramhavar::And we wanna put some funding towards kinda boosting that that work up as well.
Dina Genkina:Okay. So you mentioned neuromorphic computing is a small part of of the landscape that you're aiming to explore here. But maybe let's, let's start with that. People may have heard of neuromorphic computing, but might not know exactly what it is. So can you give us the, elevator pitch of neuromorphic computing?
Suraj Bramhavar::Yeah. My my translation of of neuromorphic computing, and and this may differ from person to person, but my translation of it is when you kind of encode the information in a neural network via spikes rather than kind of this discrete values. And that modality has shown to work pretty well in certain situations. So if I'm if I have some camera and I need a neural network next to that camera that can recognize an image with very, very low power or very, very high speed, neuromorphic systems have shown to work remarkably well. There's and they've worked in a variety of other applications as well.
Suraj Bramhavar::One of the things that I'm I haven't seen or maybe some one of the drawbacks of that technology that I think I would love to see someone solve for is being able to use that modality to train large scale neural networks. So if people have ideas on how to use neuromorphic systems to train models at commercially relevant scales, we would love to hear about them. And and that they should, submit to this this program call, which which is out.
Dina Genkina:Is there a reason to expect that these kinds of that neuromorphic computing might be a platform that promises these orders of magnitude cost improvements?
Suraj Bramhavar::I don't know. I mean, I don't know actually if neuromorphic computing is the right technological direction to realize that these types of orders of magnitude cost improvements. It might be, but I think we've we've intentionally kind of designed the program to encompass more than just that particular technological slice of the pie. In part because it's it's entirely possible that that is not the right direction to go. And there are other more fruitful directions, to to put funding towards.
Suraj Bramhavar::Part of part of what what we're thinking about when we're designing these programs is we don't really wanna be prescriptive about a specific technology, be it, neuromorphic computing or or probabilistic computing or any particular thing that that has an a name that you can attach to it. Part of what we try to do is is set a very specific goal or a problem that we wanna solve, put out a funding call, and let the community kind of tell us which technologies they think can best meet that goal. And and that's the way we've we've we've been trying to operate with this with this program specifically. So there are there are particular technologies we're kind of intrigued by, but I don't think we have 1 any one of them selected as, like, kind of this is the path forward.
Dina Genkina:Cool. Yeah. So you're kind of trying to see what architecture needs to happen to make computers as efficient as brains or closer to to the brain's efficiency.
Suraj Bramhavar::And you and you kinda you kinda see this, happening in in both I mean, in the in the AI algorithms world, as these models get bigger and bigger and grow their capabilities, they're starting to introduce things that we see in nature all the time. I think probably the the most relevant example is the stable diffusion. This this neural network model where you can, you know, type in text and generate an image. It's got diffusion in the name. Diffusion is a natural process.
Suraj Bramhavar::Noise is a core element of this algorithm. And so there's lots of examples like this where they've kinda that community is taking is taking bits and pieces from or inspiration from nature and implementing it into these artificial, neural networks. But in doing that, they're they're doing it incredibly inefficiently.
Dina Genkina:Yeah. Okay. So great. So the idea is to take some of the efficiencies out in nature and kind of bring them into our technology. And I know you said you're not prescribing any particular solution and you just want that general idea, but nevertheless, let's talk about some particular solutions that have been worked on in the past because you're not starting from 0, and there are there are some ideas about how to do this.
Dina Genkina:So I guess neuromorphic computing is one such idea. Another is this noise based computing, something like probabilistic computing.
Suraj Bramhavar::Can you explain what what that is? Intriguing property. And there's kinda 2 ways I'm thinking about noise. One is just how do we deal with it? When when you're designing a digital computer, you're effectively designing noise out of your system.
Suraj Bramhavar::Right? You're trying to eliminate noise, and you go through get great pains to do that. And as soon as you move away from digital logic into something a little bit more analog, you spend a lot of resources fighting noise. Right? And in in most cases, you eliminate any benefit that you get from your kind of newfangled technology because you have to fight this noise.
Suraj Bramhavar::But in the context of neural networks, it's what's very interesting is that over time, we've kind of seen algorithms researchers discover that they actually didn't need to be as precise as they thought they needed to be. You're seeing the precision kind of come down over time. The precision requirements of these networks come down over time. And we really haven't hit the limit there, as far as I know. And so with that in mind, you start to ask the question, okay, how much how precise do we actually have to be with these types of computations to perform the computation effectively?
Suraj Bramhavar::And if we don't need to be as precise as we thought, can we rethink the types of hardware platforms that we use to perform the computations? So that's that's one angle. It's just how do we how do we better handle noise? The other angle is how do we exploit noise? And so there there's kind of entire textbooks full of algorithms where randomness is a key feature.
Suraj Bramhavar::Right? I'm not talking necessarily about neural networks only. I'm talking about all algorithms where randomness plays a key role. Now neural networks are kind of one area where this is also important. I mean, the primary way we train neural networks is stochastic gradient descent.
Suraj Bramhavar::So noise is kind of baked in there. I talked about stable diffusion models like that where noise becomes a key central element. In almost all these cases, all these algorithms, noise is kind of implemented using some digital random number generator. And so there the thought process would be, is it possible to redesign our hardware to make better use of the noise given that we're using noisy hardware to start with? Notionally, there should be some savings that come from that.
Suraj Bramhavar::That presumes that the interface between whatever hardware whatever novel hardware you have that, is creating this noise and the hardware you have that's performing the computing doesn't eat away all your gains. Right? I think that's kind of the big technological roadblock that I'd be keen, to see solutions for outside of the algorithmic piece, which is just how you make efficient use of of noise. When you're thinking about implementing it in hardware, it becomes very, very tricky to, implement it in a way where whatever gains you think you had are actually realized at the full system level. And in some ways, like, we we want the solutions to be very, very tricky.
Suraj Bramhavar::The the agency is designed to fund very high risk, high reward type of activities. And so there there, in some ways, shouldn't be consensus around a specific technological approach. Otherwise, somebody else would have likely funded it.
Dina Genkina:You're already becoming British. You said you wanted you were keen.
Suraj Bramhavar::Yeah. I've been here long enough.
Dina Genkina:It's showing. Great. Okay. So we talked a little bit about neuromorphic computing. We talked a little bit about noise.
Dina Genkina:And you also mentioned some alternatives to back propagation in your, thesis. So maybe, first, can you explain for those that might not be familiar what back propagation AI training. Currently, you
Suraj Bramhavar::you're all AI training currently used today. Essentially what you're what you're doing is you have this large neural network. Neural network is composed of you can think about it as this long chain of of knobs. Right? And you you really have to tune all the knobs just right in order to get this network to perform a specific task.
Suraj Bramhavar::Like when you give it an image of a cat, it says that it is a cat. And so what back propagation allows you to do is to tune those knobs in a very, very efficient way. Starting from the end of your network, you kind of tune the knob a little bit, see if your answer gets a little bit closer to what you'd expect it to be. Use that information to then tune the knobs in the next the previous layer of your network. And keep on doing that, iteratively.
Suraj Bramhavar::And if you do this over and over again, you can eventually find all the right positions of your knobs, such that your network does whatever you're trying to do. And so this is great. Now the issue is every time you tune one of these knobs, you're performing this massive mathematical computation. And you're typically doing that across many many GPUs. And you do that just to tweak the knob a little bit.
Suraj Bramhavar::And so you have to do it over and over and over and over again to get the knobs where you need to go. There's a whole bevy of algorithms. What what you're really doing is kind of minimizing error between, what you want the network to do and what it's actually doing. And if you think about it along those terms, there's a whole bevy of algorithms in the literature that kind of minimize energy or error in in that way. None of them work as well as back propagation.
Suraj Bramhavar::In some ways, the algorithm is beautiful and extraordinarily simple. And most importantly, it's very very well suited to be parallelized on GPUs. And I think that isn't is part of its success. But one of the things I think both algorithmic researchers and hardware researchers fall victim to is this this chicken and egg problem. Right?
Suraj Bramhavar::Algorithms researchers build algorithms that work well on the hardware platforms that they have available to them. And at the same time, hardware researchers develop hardware for the existing algorithms of the day. And so one of the things we want to try to do with this program is blend those worlds and allow algorithms researchers to think about what is the field of algorithms that I could explore if I could rethink some of the bottlenecks in the hardware that I have available to me, similarly with the with the in the opposite direction.
Dina Genkina:Imagine that you succeeded at your goal, and, you know, the program and the wire community came up with a one one thousands compute cost architecture, both hardware and software together? What does your gut say that that would look like? Just an example. I know you you don't know what's gonna come out of this, but
Suraj Bramhavar::Yeah.
Dina Genkina:Give us a vision.
Suraj Bramhavar::Similarly, like I said, I I I don't think I can prescribe a specific technology. What I what I can say is that I can say with pretty high confidence. It's not gonna just be one particular technological kind of, like, pinch point that gets unlocked, it's going to be a systems level thing. So there may be individual technology at the chip level or the hardware level. Those technologies then also have to meld with things at the systems level as well and the algorithms level as well.
Suraj Bramhavar::And I think all of those are gonna be necessary in order to reach these goals. When I I'm I'm talking kind of generally, but what I really mean is, like, what I said before is we gotta think about new types of hardware. We also have to think about, okay, if we're gonna scale these things and manufacture them in large volumes cost effectively, we're gonna have to build larger systems out of building blocks of these things. So we're gonna have to think about how to stitch them together in a way that makes sense and doesn't eat away any of the benefits. We're also gonna have to think about how to simulate the behavior of these things before we build them.
Suraj Bramhavar::I think part of the power of the digital electronics ecosystem comes from the fact that you have Cadence and Synopsys and these EDA platforms that allow you with very high accuracy to predict how your circuits are gonna perform before you build them. And once you get out of that ecosystem, you don't really have, have that. So I think it's gonna take all of these things in order to actually reach these goals. And I think part of what this program is designed to do is kind of change the conversation around what is possible. So by the end of this, it's a 4 year program.
Suraj Bramhavar::We we wanna show that there is a viable path towards this end goal. And that viable path could incorporate kind of all of these aspects of of of what I just just mentioned.
Dina Genkina:Okay. So the the program is 4 years, but you don't necessarily expect, like, a finished product of a one one thousandth cost computer by the end of the 4 years. Right? You kind of just expect a a to develop a path towards it.
Suraj Bramhavar::Yeah. But we I mean, ARIA was kinda set up with this kind of decade old time horizon. We wanna push out we wanna fund, as I mentioned, like high risk, high reward technologies. We have this kind of long time horizon to think about these things. I think this the program is designed around 4 years in order to kind of shift the window of what is what the world thinks is possible in that time frame.
Suraj Bramhavar::And in the hopes that, you know, we change the conversation. Other folks will pick up this this work at the end of that 4 years, and it will have this this kind of large scale impact on a decade
Dina Genkina:Great. Well, thank you so much for coming today. Today, we spoke with doctor Siraj Bramhavar, lead of the first program headed up by the UK's newest funding agency, ARIA. He filled us in on his plans to reduce AI costs by a factor of 1,000, and we'll have to check back with him in a few years to see what progress has been made towards this grand vision.
Dina Genkina:For IEEE Spectrum, I'm Dina Genkina, and I hope you'll join us next time on fixing the future.