Conversations between the venture investors and operators at Dell Technologies Capital and the people who are building what's next in enterprise technologies.
Intro
Those kind of deep relationships are the ones that are going to have long-term impacts on the platform. And then we're going take those long-term impacts and then educate everybody else about them. That's how we build this flywheel. We kind of curate the early adopter innovator, the people that we really believe in. And then we take that learning and that pain and we turn it into a product that more people can use to be successful.
Welcome to the DTC Podcast, a series of conversations between the investors and operators at Dell Technologies Capital and the people who are building what's next in enterprise technologies. Hello and welcome to the DTC podcast. In this episode, DTC investor, Radhika Malik talks with RunPod co-founder and CEO, Zhen Lu, to cover off on Zhen's mission to build a platform that helps developers take AI-driven projects from the tinkering phase all the way through scale. This is a process that Zhen knows a little something about personally, having built RunPod from day one to a Forbes Next Billion Dollar Startup in just two short years.
Interview
[Radhika] Let's jump in. I am very excited to welcome our portfolio founder, Zhen Lu. Hey, Zhen, thank you for coming. Very excited for the conversation today. Thanks so much for the invite. Awesome. So let's jump right in. What are you building with RunPod? I'm going to pretend I haven't heard that before.
That's a million dollar question, right? We are building a platform that is targeted towards developers. That is very broad. Specifically, we are targeting developers that are interested in AI inference. I believe that one of the things that I've seen our developers do a lot is mess around, right? There's a lot of cool things coming out of the market every day, every week. And what's also obvious to me is just the amount of pain out there when it comes to how to make this stuff work. And as somebody that believes in lowering the barrier of creation, it's something that resonates very deeply with me because I don't think that the barrier to creating these cool things should be, what type of GPU do I pick? Or what kind of inference engine do I pick? Or how do I string this stuff together? It's really, well, how do I fine tune my model in a way that's going to offer unique experiences? How am I going to add my taste, if you will?
And so, the platform that we're building is really geared towards how do we get these people, these developers, the tools needed so that they can focus on adding their uniqueness, their taste to what they're building. Because I think that there's a lot of crazy stuff that is getting developed. And I also see that there's just a lot of time wasted, right? So how can we reduce that amount of time? That's what I'm really interested in. That's awesome.
[Radhika] As a former developer myself, I can attest we like messing around. What does messing around and enabling developers to mess around mean for you guys specifically? You touched on it a little bit with fine-tuning and inference and this new technology that we all call AI and enabling them to do more interesting things with it. But maybe give us some examples or share some anecdotes on what that messing around in the context of RunPod means and how you think about enabling that.
[Zhen] Yeah, absolutely. And maybe I can anchor that in a little bit of a story, right? Where when we started RunPod, I won't lie to you and say, hey, you we had this grand vision of exactly what the road would look like. Now, we started with this idea that we thought that there needed to be better tools. There needed to be some analogs to traditional cloud. And we were going to set out to build those tools. However, we kept a really open mind. And so we built something in the beginning, our v0, if you will, of RunPod.
And it was entirely geared towards allowing more individuals, right, prosumers, people to mess around, right, get in and just try things. And that was really awesome because people built some really, really interesting things and they shared it with us and we got to experience that with them. However, very quickly, as we started getting more and more users, they started saying, hey, you know, we messed around with this, right, we fine tuned a model or we got it working, right? It works. Like here's a POC, but like how am I gonna actually make this a real thing? How am gonna make this into a real product? How am I gonna scale this? And now you get into like the, sure you can make it work on one GPU, one computer. How can we make it work on 10 or 100 or a thousand or 10,000? Because one GPU can only serve a handful of users. And we're expecting the people that use RunPod to have large outcomes. If they wanna have large outcomes, they need to be able to scale. And so coming back to the question, right? How do we do this? What are we most interested in? It's pretty easy to go somewhere. And if you know what you're doing, or you do some Googling, watch YouTube video, you can get pretty good at fine tuning a model. And that's fine, right? Okay, then once you're done fine tuning the model, let's say RunPod gives you the environment to do that.
What next? Where do you put it? How are you going to move that somewhere where you can actually do something with it? And the next thing that you would obviously want to do is, well, try it out, even if it's just for yourself. Evaluate it. Is it good enough? If you decide it's good enough, am I going to upload a dogging phase? Am I going to download it to my local machine? Am I going to put it in a cloud bucket somewhere? And RunPod is the platform that should be able to tell you, OK, this is the next step. Once you fine tune your model, now you can do this.
Here is where you actually go and test it. Here are the tools for you to figure out whether this is any good. And once you figured out it's any good, here's the way you actually deploy it. And now, hey, maybe you need to think about other models as well. Maybe they need to work together. Here are easy ways for you to click those things together. And we're not going to limit your ability to actually add what you need to add and also to be able to scale it. So there's this idea that we've been talking about around, I think there are a lot of companies out there that are trying to enable copying, right? Somebody is going to go and create something for you and you're going to copy it. And then you're going to try to wrap that in something. And we don't really believe in that. I think that there is value in being able to copy. But I like this idea of forking more, right? In software, how do you get started with stuff? Well, you can maybe go find software that's out there and you can fork it and you can make it your own. But that's not trivial and that doesn't come with most products. And that's something that we are really interested in enabling.
[Radhika] That's really cool. And I remember whenever you started this company, think it was like a year and a half ago or something, not too long back, right?
[Zhen] Yeah, we're two and a half years now. It's been a while. It doesn't seem like as long because in the beginning, Pardeep and I were just trying to make it work. It didn't really feel like we'd started a company.
We're just like, hey, we're doing this cool stuff and we've got users. No, hey, we're making money. So, you know, we technically started the company in March of 22, but it feels both much longer than that and much shorter than that because of the way the company has grown.
[Radhika] Yeah, absolutely. It's been an incredible ride for you guys and a really exciting time. When you guys actually started, I know there was Stable Diffusion and people were doing all sorts of creative things with it, fine-tuning it, generating all sorts of crazy images. Did you see some crazy stuff happen on the RunPod platform when you enabled that and that whole wave started?
[Zhen] Yeah. It's funny, right? Because when we started RunPod, the Gen.ai hype cycle hadn't really even started yet. We knew that there were cool things being done and we were tinkering around with it as well. And that was just really awesome. we were like, we were kind of a little bit early, right? Which actually worked out for us because we were able to battle test with actually a lot of our early customers were the creative type. They weren't developers. They were like mathematicians. They were artists and physical artists, not even digital artists. I remember one of my early customers that I worked quite a bit with in the Disco Diffusion days, if you can imagine. Disco Diffusion, precursor to stable diffusion. It took minutes to generate things. And I did a lot of work working with these creatives to figure out how they could actually scale their more manual workloads. Where they're like, hey, I need to generate like 1,000 images so that I can then do digital art on top of it or even physical art on top of it. How do I do that? Because I have one GPU, it takes minutes to actually generate one image. Can you help me out? So I did a lot of scripting with them and teaching them.
And one of my early customers, he must have been like 70 years old. 70 years old barely knew how to use a computer and he was on RunPod trying to figure out how he could batch generate images with disco diffusion. And I'm not going to say that it was easy to work with this person and get him running, but it was just, that is how compelling this technology was and how motivated some of these people were to just like get going. And it was just like some of the coolest interactions that I've ever had where person doesn't even know what like a terminal is. I'm like, okay, you go into the terminal, open here, you click here, this is what you need to do. And then just seeing the output, the final output of his physical art was just amazing to me. It was like, this is like the actual value that is being created. And continuously kind of over the lifetime of RunPod, we've been able to kind of see that value evolve over time. And that's been very exciting. Yeah, tell me more about that.
[Radhika] How has it evolved over time, two and a half years now into building the company? From just the two of you, now the platform, the AI way it came, as you said, large language models became a thing, ChatGPT came and changed everything. How do you think about the platform now and what it means to foster creativity with this wave that's ongoing?
[Zhen] Yeah, I would say that when we first started the company, as these things go, you can't really be super picky, right? And then whoever comes is who comes and you try your best to learn from them and serve them. And over time, as our user base has grown, we've had to become more picky in kind of who we're serving, not that we're turning people away, but we believe that we need to focus in order to really provide a compelling experience for our target audience. So we started zeroing in on: Do we like the creatives? We love them. Are we really building a platform for them? I mean, really not. We're building a platform that's more focused for our current definition of developers. Now, I do see that definition evolving over time, and we will have to evolve over time with it. But right now, there are people that need to write code. We want to take away as much boilerplate as possible. We want to abstract away as much of the infrastructure and networking as possible, because we don't believe that's what the value is, right?
The value that the developers are going to be bringing to the products they're going to be building on our platform is really their ability to add their taste, their ability to compose these models, their ability to really be creative in the space rather than their ability to pick the right GPU or to figure out what the best way to scale, right? The scaling algorithm or the best way to figure out what the retries are, or any of that stuff, right?
And so over time, we have seen that we've successfully been able to move more towards a developer-focused community and also move more towards a production-focused community, where I love the tinkering, right?
You need to do the tinkering in order to get to the production. But once you're in production, that's where you can actually start generating the value. And so, creating a platform where the tinkerers can share their learnings, not only with the community, but with us, so that we can create the platform that allows the people that want to go to production and actually serve people and deliver value. That's kind of what we've been focusing on. And so, we've migrated more towards small teams of developers. Some of those teams are operating in larger companies. Some of them are startups themselves. And it's just been really exciting to see the diverse set of groups that have been using us. And they all share this small teams that are trying to achieve large outcomes.
[Radhika] That's great. What are in your mind, some of the most interesting and or successful projects that people have built in production with RunPod today?
[Zhen] Yeah, so I'm not really at liberty to share specific names. However, there are quite a few media kind of consumer applications that have been really successful. I think some of them actually even hit like top 10 on like Apple Store or the Android Store. These are really interesting to me because they are kind of the epitome of needing to provide taste because their customers come to them because they have novel experiences. They come to them because they are enabling the end user to kind of see themselves and see each other differently, right? So they're providing different AI models that can run and augment photos, right? And the only way that they're able to stay successful is they're letting RunPod take care of the heavy lifting of all the infrastructure and scaling and stuff. And so they can focus on adding their taste and evolving that over time because their customers aren't going to be happy with a boring platform that doesn't change over time. They have to continually add more and more taste, add more and more of what their customers want. And so they've been very thankful to RunPod for offering them a platform that can grow with them in that way and take off that kind of more boilerplate scaling workload from them. Another type of company are the, we actually see quite a bit of what I would say augmented reality kind of platforms where, for example, real estate staging, right?
Now, I've to sell a few houses and generally your houses sell much better if you can pay somebody to actually put furniture in them and stage the home. So then when people come, it's like, you know, I can see myself living here, right? It's not just like an empty box. Most people don't like to look at empty boxes. They like to look at what can my life look like in this house? And so quite a few companies have come to RunPod and are running models that allow virtually to stage.
And that's been incredibly successful for them because there's a huge need for that kind of company. And then again, it's how well can you not only train a model to do that, but then put together the pipeline, right, to make something that's compelling because I've definitely looked at photos and I'm like, yeah, that's totally AI generated. But then I've looked at other photos and I'm like, wow, this is like real, I can't tell. Like I feel like it looks really good and I kind of want to say, that's AI generated, but I'm like, no, this might be real. And so we're really hitting that threshold of you can get some really, really good results from this. And that's really awesome because as somebody that will likely have to sell more houses in the future and buy more house in the future, I would like to not only just get one person's view of how their house could be staged, right? They paid somebody, they staged it. I would love to be able to say, okay, here is a house, what does my furniture look like in it? What does it look like if I wanted to put my kind of thoughts and style into it? And so to be able to kind of tell more of those stories in a very compelling way, I think that's very, very interesting to me because I don't know about you, like I want to live how I want to live, right? I don't want to live how somebody else tells me how to live. And, you know, one of those companies seems like they've gotten recently a pretty big outcome, right? So again, along this theme of allowing small teams, right? These teams are like 10, 20 people, to be able to achieve these really huge outcomes and to be able to serve so many customers and to be able to impact the world in how I feel is a positive way, that's incredibly motivating to me.
[Radhika] That's incredible. Absolutely. How did you have to evolve the platform to go from a single developer, prosumers, to teams of developers, to actually teams of developers running pretty big applications at scale, reliably, but still letting, still embracing that experimenting, iterating mindset that seems to be a little bit at odds with each other. But how did you think about the platform from a first principles perspective and what that means?
[Zhen] Yeah. So when Pardeep and I started RunPod, we were not people with deep pockets and we didn't really want to go and honestly had no experience going and raising a ton of money so that we could like build a data center. So when we launched RunPod, we actually launched it off of some servers that we were hosting in our basement. So Pardeep actually got electrical work done on his home so that he could turn on these servers and they wouldn't just blow his electricity. I did not get electric work done, but I did have to finagle what circuit breakers go to what outlets so I could plug in these servers without doing the same. And so in the beginning, we definitely came from very humble beginnings. We were able to serve the people that didn't really care. They wanted to come to us because we had a good user experience.
They wanted to come to us because they believed in trying new things and we were eternally thankful for them to give us a try. And then as we got more and more customers that were interested in this and the feedback was incredibly positive, we had to find other people that were also running hardware in their basements or in sheds or wherever.
And as we started moving towards more business use cases, that wasn't going to fly. So then we had to evolve it to, okay, now we have to go and make some partnerships with data center operators and really hold ourselves to a higher standard.
So then, we started this idea of separating community cloud versus secure cloud. Where secure cloud, we could offer more of those guarantees. It's a higher level of infrastructure. The networking is better, right? The GPUs are better, right? Everything is just held to a higher standard. And so that's been a journey.
And then also on the software side, the platform itself that our customers interact with, that's also been a journey because in the beginning, it's like, OK, you make an account. You can mess around with stuff. And then people are like, OK, so what if I want to collaborate with somebody? And we're like, yeah, you should probably be able to do that. So let's make it so you could do that. So then we had to create team accounts. We had to make some role-based access control. We recently rolled out API key scoping.
Because in the beginning, you got an API key, gave you access to everything in your account. And people were, as they were moving in bigger teams, they were like, but what if I have to off-board somebody from my company? And they have our API keys. We can't rotate them. Or they have access to the entire accounts. We're like, OK, yeah, fair enough. So, adding more and more granular control while not making it so complicated that it's like you have to learn an entire programming language or entire system to be able to use it has been kind of the way that we've been balancing it.
And I guess to the point of how do we balance like the tinkerers versus production? You know, it seems like it might be at odds with one another, but to us it's like, it's the same like story, right? It's on the lines of we don't think that there's value in just offering the tinkering platform because what are the holes that we see is like, how do you get from tinkering to production? And that's actually the thing that we want to solve.
Because the people that are in production, mean, honestly, those are a few and far between. There's a lot of people that want to bridge the gap. How do I go from tinkering to production? Because I can tinker around with it, and I can build something that kind of like works for now. But there's no guidance in terms of, well, how do I bridge the gap? And I think that there's a lot of the activation energy barrier there, where I think a lot of people give up. Like, if that's something, I think it's really cool. But like, I have no idea how to actually do anything with this? And I see RunPod as being the gap filler here. Yeah, that's what I was going to ask you.
[Radhika] Where do you think the gaps are with the cloud platforms today that don't allow you that spirit of tinkering or that experimentation spirit or maybe the gap between going from experimentation to actual productionizing it? Because I've also heard you talk about, hey, I'm building the cloud I wish I had as a developer. So where do you think the gaps are and why do you think those gaps exist?
[Zhen] One of the gaps is just that nobody honestly really knows what they're doing. And so people are looking for guidance. And I think that there are some platforms out there that are afraid to give that guidance. And so it's too open, right? And these are kind of platforms that are just going to offer you the raw compute and say, well, I hope you know what you're doing or like it's your job to figure it out. And maybe I'll have some docs or some guides that are like, here's how other people are doing things, but we're not going to be opinionated, right? This is not how you should be. It's just like, this is a way that you could be doing it. And then there are other platforms that I feel like are jumping the gun, maybe making their opinions too early, where they're like, this is the only way you can do it. Take it or leave it, right? And I feel like we're too early for that, but we're far enough along that we shouldn't be just throwing GPU servers at people and saying, I hope you know what you're doing, because the reality is most people don't.
So, the way I see RunPod is we build the community and we obviously have opinions on how we like to do things, but we're going to test them against our community and adopt them in a more measured way. And so that is stuff like how do we see infrastructure as code playing out, right? Where right now we're no longer in like kind of the web era where the artifacts that you're most interested in is really small bits of code. We're now in a place where it's like, OK, well, the code is still important. And the code is actually kind of more the architecture level code. There's code at more different levels. Plus, now you have models that are pretty huge. Even the small models are huge. And you've got multiple models, right? So now you have to carry around a lot more baggage. And the way that people are going to be asked to say, hey, this is how you should store your artifacts. This is how you should store the relation to one another, and this is how you can interact with them. That's, think, the opinion that really needs to be out there. And that's what I think that most developers need guidance on. Otherwise, they are just kind of like, well, cool. I can deploy something that offers maybe me some value on my local machine. But it's not clear to me how I'm going to actually build something that's much more robust than that.
[Radhika] Yeah, that's fascinating. It's like, you want to be opinionated enough so people can get value and you can direct them in the right way to use technology to again, know, harness their creativity and add taste to it. But if they do want to get deep enough, you want to still be flexible enough on the platform to let them do more interesting things as they get more acquainted with the platform and start doing more interesting things with it. So it's a pretty interesting product and experience challenge as I, how you think about it for developers.
[Zhen] Yeah, and I think that there's the one of the opinions, is actually where to have an opinion. Because, right, there's some things that I'm like, okay, it should be locked down. Because guess what, right? Like, even if you're going to do some stuff here, it's not going to offer that much value. Like it might make you feel good. However, it's not going to really be where you should be adding value. So like one of our opinions is actually, this is what's locked down, right? You don't get to touch this because even if you might want to, you probably should be spending your time elsewhere. Here is where you spend your time, and here are the tools that are going to help you spend your time in a more efficient way.
[Radhika] And how do you leverage customer needs or user needs or developer needs to inform product decisions? And where do you say, hey, these guys don't know what they want. They're incredibly smart and they want to add taste to their products. But from an infrastructure perspective, we want to listen to them and really address their needs. But at some point also, be opinionated about where we should have opinions again and where we shouldn't? How do you balance those two? Because developers can be really opinionated themselves as well.
[Zhen] Yeah. I would say that the way that we've been approaching it is we have been fortunate enough to have quite a lot of users use our platform. So there is a bit of like a strength in numbers. And I'm not going to say that we make all of our decisions based on like how many people are asking, but that does allow us to pick and choose who we feel are the most successful, who we feel resonate with us the most. Because at the end of the day, right, we are kind of looking for the kind of customers that we feel are gonna be most successful. Because we wanna be ultimately a platform where successful companies build, right? Successful groups of people are going to actually be able to achieve, right, their dreams.
It's funny because I feel like sometimes I need to act like an investor, right? We're investing our time and energy into the kind of futures of our customers. When they're successful, we're successful. And if they're not successful, we won't be successful, right? We're not trying to build kind of a flywheel of just churning people in and out, right? That's not going to make us ultimately successful. So, we are choosing, like, hey, what is your use case? Talk to me about it. How is that going to be differentiated?
How are you going to spend your time and differentiate from the market? And if we can help you with that, that's great. And so, we're trying to be strategic about how we're going to build this product in a way that's not just looking at the next week. When we started RunPod, we did a lot of that, where our customers would ask us for things. the best feeling would be a customer would ask me for something. And I'm like, I have the code in my IDE. I just haven't pushed it to prod. Let me do that for you right now. And then they're like, that's amazing, right? It's like magical. I asked for it and I got it like almost in real time. And now we're moving more towards a measured, well, what are we going to deliver, right, in the next several months or even like next year that's really going to move the needle for some of these customers.
[Radhika] And how do you do that? Given one of the big strengths was the community and the developer love that we bought early in our days, ready through word of mouth and developer sharing experiences with each other. Because as you said, you guys were on a shoestring budget, so I don't think you were spending money marketing or doing anything of that sort. So, you got a lot of developer love. How do you think about continuing to nurture that love, but still being selective in who you actually choose to make successful on your platform? And truly we thought partners with them.
[Zhen] Yeah. So the way I see it is there are the things that scale, right? I think the things that scale are things like us being able to just offer credits to people, right? Like we have programs where people can come and they're like a startup, they can apply, right? They can get some credits. It's pretty low touch. And then there are the things that are even more widespread. Like we can publish blog posts and guides so that more people can try things out, right? And really push their boundaries in terms of what they're able to do. But there are things that like don't really scale as much. And those are more of the things that are like well, we're gonna invite like some of our top customers that we believe are creating differentiated projects to actually come in and spend time with us, whether that's in person or on a call, where we really dig deep into their use cases. We dig deep into what pains they have. So then we can really decide, okay, what are we best equipped to actually help them with?
And I think really those kind of deep relationships are the ones that are gonna have long-term impacts on the platform. And then we're gonna take those long-term impacts and then educate everybody else about them. And I think that's kind of how we build this flywheel of we curate the kind of maybe the early adopter, innovator, the people that we really believe in. And then we take that learning and that pain and we turn it into a product that more people can use to be successful. And then we kind of turn that flywheel as much as we can until we get to something that we feel is mature enough. And then we can approach things a little bit differently.
[Radhika] Yeah, hopefully if they build something with us that they're afraid of, they will go and share with folks and continue to build the community. And as you said, the flywheel will continue to keep growing as more and more interesting projects come in and people come in, share their learnings and share how they used our platform to build cool things.
[Zhen] Yeah. I think the storytelling is incredibly important. I think that there are people out there that really just need to, they need to kind of believe. They need to believe that there are small groups of people or even individuals that can go and build these really exciting products. And the more we can tell that story of, hey, RunPod can help you do that, or it's at least possible. Like it doesn't even need to be RunPod, but this is possible. Go do it. I think the better off we're going to be.
[Radhika] That's great. So we have talked about all of the positives. There's a general concern from the community, AI community in general, that there's been a lot of hype around AI and the ROI, just the return on all of the GPU expenditure, all of the other infrastructure expenditure hasn't really played out so far. There's been a lot of, in our words, tinkering, but not as many production applications and true value out of AI. How do you think this plays out from a macro perspective?
[Zhen] Yeah. I mean, I will 100 % agree that we are in some kind of bubble. I haven't really met too many people that aren't going to agree that we're in some kind of bubble. It's just a matter of how big and when is it going to burst? And I think the thing that I'm kind of most cognizant of is how much of the current value that's being created is actually being paid for. I think that's the thing that worries me the most because I don't think anybody can argue against AI, Gen AI specifically, providing some real value. I mean, it's already changed lives. It's changed the way that we work. It's changed the way that we think about things. That's not going to go away.
I think the question really is, who's paying for it? And can they continue to pay for it? And is this sustainable? And so what I'm looking for is I'm deeply interested in how the compute efficiency is going to scale over time. Because I think if we can continue to kind of drive down the cost of compute, which I think is a good thing, then we can start valuing the things that actually are going to have an impact. And to me, that's less about the compute and that's more about what we do with that compute, right? What are the actual use cases? How are people actually going to use this to change what they're doing on a daily, right? How is it going to make it into the hands of like, you my stepmom, like for example, right?
And these, think these are the kind of questions and are we going to be able to get there before people realize or like, okay, well, we're basically running off of a lot of VC and subsidized money, right? It's not just VCs, it's actually hyperscalers as well. It's all the people that are interested in making this thing go. And I feel like there's eventually going to be at least a little bit of consolidation here, right? Where the companies out there that are not actually creating value, I think there's talk of like the grifters, right? They're gonna die. The companies that have been too, I guess, aggressive, right, in land grabbing, I think that they're gonna die. But I think that the companies that are really focusing on how they're producing real value and really focusing, right, on that value in a kind of a more sustainable way, I think those are going to be the ones that succeed.
[Zhen] With that backdrop, last question. What's next for RunPod? What's next for RunPod? Well, we are definitely doubling down on making sure that we can have really deep interactions with our current customers. Like pretty much every company, right, we're always interested in how we can be better, right? How we can be better in terms of our developer experience. How we can offer an even more reliable platform, right? Can we get more nines onto our reliability so that we can increasingly support larger scale?
I would say that we have done a really good job of supporting some pretty massive scale already, but my vision of kind of an AI world is that this will only grow and it will grow at an insane pace and it's gonna be challenging, right, to actually keep up with that.
It's a challenge that I'm really excited about.
[Radhika] Awesome. Well, thank you so much, Zhen. This was a fascinating conversation. I wish we had a couple hours more to keep talking about some of these topics. I really appreciate you being here and thanks for the time.
[Zhen] Thanks so much, Radhika. Thanks for listening to the DTC podcast. If you like what you heard, you know what to do. Like, share, and subscribe wherever you get your podcasts.