Show Notes
About Bret McGowen:
Bret is a Developer Advocate for Serverless on the Google Cloud Platform team at Google, focusing on serverless products like Cloud Run, Cloud Functions, App Engine, Firebase, machine learning APIs, and more. He's currently an aspiring Node.js developer. Prior to Google, Bret worked as a software engineer in the cloud industry at Rackspace. Before that, he led engineering teams at e-commerce and non-profits and tried his hand at creating two startups.
Transcript:
Jeremy: Hi, everyone? I'm Jeremy Daly, and you're listening to Serverless Chats. This week, I'm chatting with Bret McGowen. Hi, Bret? Thanks for joining me.
Bret: Happy to be here. Thanks for having me.
Jeremy: So you are a developer advocate for serverless at Google Cloud. So why don't you tell the listeners a little bit about yourself and sort of what's going on with Google Cloud?
Bret: Yeah, absolutely. So my background is I was a software engineer for a long time. I worked at another cloud company called Rackspace as a software engineer, working on some of their networking stuff. I kind of have a long history of being a full stack developer from e-commerce platforms to startups and stuff like that. In one of my previous roles at Rackspace, actually, we started working on tools for developers to actually consume the Rackspace cloud, so that was doing things like writing tutorials and guides and useful videos and giving talks to teach people about it. Then it turns out there's actually a whole job that does that full time, and so that's kind of what led me into being a developer advocate at Google.
Jeremy: Great. So Google Cloud, I think most people know what Google is. But Google Cloud is a public cloud provider, right?
Bret: Yes. So Google Cloud is kind of an umbrella term for... There's sort of two halves to it. One is what used to be called or is called G Suite, and so that's like Google Docs and Gmail and Google Sheets and things that your business uses and kind of your frontline knowledge workers use to put stuff in spreadsheets and send emails and all that. So that's the G Suite half of Google Cloud. Then the other half of Google Cloud is Google Cloud platform, and that's like the public cloud databases and virtual machines, if I'm allowed to say that in your podcast and serverless tools and things like that, right? So, when you build your application, these are the tools you use as a Google Cloud platform. That's what I focus on, serverless specifically.
Jeremy: Awesome. All right. So, in terms of the developer advocate position, because you hear developer advocate and developer evangelists and all these different things depends on which cloud or which provider you're talking to. So I know these roles are a little bit different at different organizations. So what exactly is a developer advocate at Google Cloud?
Bret: Yeah. So we have sort of two jobs. The first people probably are more familiar with because it's a little more public facing, and that's we do things like I was just talking about, go out and teach people about Google Cloud, how to solve problems. But kind of more broadly than that, just to help developers be successful, so whether that's on Google Cloud or not. Our mandate is a little bit different, right? It's to help developers solve problems that they're having, and maybe with Google Cloud, but maybe not. So we do that, again, through blogs and videos and in person talks and customer meetings and stuff like that. Then the other half, which I think is maybe less common, but super, super important is to be a funnel of input from the community into the Google Cloud platform team. So I work with the product and engineers to help deliver feedback, like, "Hey, here's what I'm seeing that folks are struggling with when they use certain tools," or, "Here's the categories of problems that I see people trying to do with serverless."
I go to a lot of events and talk to a lot of folks, and so kind of being that funnel of input. So part of my job is to hear, really specifically on Google Cloud, what works well and what doesn't, but also, hey, maybe I tried using Google Cloud, and I can't because this whole features isn't working the way that I expected to, or this is completely missing, or my workload isn't a good fit for this reason. So that's part of my job. So really want to make sure you're listening, know if they have questions about Google Cloud or they want to see things changed, like please, please let me know, happy to hear it. A lot of times, I'm maybe saying some of the same things already, but it always helps to have someone who's a customer or potential customer that carries weight because that usually also carries dollars.
Jeremy: Awesome. All right. So I have seen you speak about this thing called Google Cloud Run a number of times. I'm sort of in the AWS ecosystem. That's where I'm the most comfortable with. But I got to say Google Cloud Run looks awesome, and I really like how it's sort of this different approach to serverless. So maybe you could give the listeners an overview of what Google Cloud Run is just so they know what it's all about.
Bret: Sure. So Cloud Run is a serverless platform on Google Cloud that's based around containers. We'll go into a little bit more about what exactly a container is, but the short version is it lets you move a lot more of your existing workloads into a serverless environment. I know a lot of us, we're working at jobs where we'd love to start brand new projects and do everything with FaaS functions first and really make this cool, awesome distributed system. But the reality is, a lot of us work at organizations that have existing products and services that maybe we wrote before serverless was a thing. So containers basically let you specify what kind of run time, what kind of libraries, what kind of language, whatever you want in a very standard way. So, if you can take a lot of these existing applications, especially if they're stateless and wrap them in this container.
So you can move existing workloads onto a serverless platform without having to rewrite everything as functions as a service. If you're a purist, whether the fully serverless or not, I don't know, but I think we all can agree we want to move more workloads, more serverlessly as quickly as possible, and so Cloud Run, I think, really opens up a whole set of new workloads that you can run on serverless, outside of just moving pieces one at a time to serverless functions.
Jeremy: I think people hear containers, and they get a little bit... They think that's starting to move away from serverless. Under normal circumstances, I would tend to agree with that because you're talking about using some sort of orchestration system to manage all that. We can talk about Kubernetes in a minute. But what I really like about Google Cloud Run is it's serverless containers, and I know that seems kind of strange, but maybe you could explain that a little bit more.
Bret: Yeah. Maybe we should have done a warning at the beginning of the podcast that, I sometimes say this in my talks, we're going to use some words that may be offensive to some of your listeners. So we're going to say Kubernetes. We're going to say virtual machines. We're going to say offline. We're going say infrastructure. So just a word of warning. I don't know if you can bleep these out on your podcast. No, just kidding. Yeah, so I think that there is kind of always been this tension in the serverless community with containers, and I think part of that is due to the popularity of Kubernetes. Kubernetes, as you mentioned, is the orchestration system for containers. But part of the problem is Kubernetes is not meant for developers.
It's super popular and has a lot of sort of buzz word behind it, but it's designed for sort of Ops and the DevOps and the SREs of the World to make their jobs easier, to help run and orchestrate and do things like health checks and rescheduling your jobs to different machines, to different nodes when they die and their problems. But containers, while Kubernetes runs containers, they're not synonymous, right? You can actually just think of containers as a packaging mechanism. It's just a way to sort of declaratively say, here's what my run time environment looks like. Here's the base operating system. Here are some certain libraries I want to install. I want this version of Node.js, and I want these 3D rendering libraries included or whatever.
I guess maybe to take just a quick step back to talk about what are containers. Just really briefly, obviously this is like a whole deep topic of discussion. But it's sort of like a really lightweight, declarative way to package or run your apps, like a virtual machine, but it is not a virtual machine. So it essentially compiles your application, so your application code and kind of the run time you described with it into this deployable unit, and so you can deploy it to your laptop, to staging, to production. Then includes, again, not just your code that is running, but also the underlying operating system and dependencies. So it's really, really reproducible and really, really portable because you're actually executing it in sort of this "virtualized environment" that's abstract or that's isolated from the actual environment that it's running in.
So I think to kind of bring it back to how does serverless and containers relate. Because a lot of the tension was running containers and Kubernetes is not serverless. So how can we actually put those together? I think one way to think of it is, when you're, your functions as a service, almost no matter what platform you're running on, it's going to take your code, and it's going to create a container out of it. That's actually the unit behind the scenes that your cloud provider or your Ops team is using to scale it up and down. Now, you as a developer may never see it. You may never know or want to know anything about it. But if you think about it, why not also give their developer as ability to just give the container directly, and that way, you can customize it to look exactly how you want as a developer or an operator.
I actually think they can play really nicely because it's just sort of skipping that first step where your cloud provider or your tooling is creating the container for you, and you can actually create yourself, if that makes sense, right? I don't want to say doing containers is a better way, it's just for some people, it is. There are a lot of benefits with portability and independence. I talked to a lot of developers who are really concerned about vendor lock-in and things. Containers is an open standard. There's a huge ecosystem out there around building containers from every major CI/CD system. There are published gold container images from certain organizations. There's an official Python image, official Node.js image. There's all these official published images that you can kind of inherit from, start from, and you know that that base image has been tested and vetted and sort of approved. So there's a really vibrant ecosystem around containers. But to finish my point, it's not necessarily better for everyone. There are some advantages, pros and cons to everything. You give up a little bit more ease of use, but you gain some flexibility and control in response.
Jeremy: Yeah. I think that when it comes to FaaS and whether that's Lambda functions or Azure functions or even Google Cloud functions, there is that really nice sort of, I don't have to do anything, I just have to write a little bit of code, put that in there, and then it sort of does everything for me. But you run into limitations with that. I know I run into limitations of that where you want to use some sort of package or some sort of binary that you want to run behind the scenes. It's getting a little bit easier with Lambda, with layers and some of that other stuff there. But sometimes, if you want to bring it down a level and you want to have the ability to sort of package and run very specific applications, containers allow you to do that.
To go back to that whole serverless thing, this is not like Fargate, if people are familiar with Fargate, where you're still sort of scheduling containers. But when requests come in, these spin up just like Google Cloud functions or Lambda functions or Azure functions. So this isn't about having to schedule a whole bunch of them to run. This just happens automatically for you.
Bret: Yeah, exactly. I think one of the things that maybe we can talk a bit about is a lot of people focus on of what's different, like how serverless containers is different than FaaS. But there's actually, in my opinion, probably more in common more that's not different, right, that's the same. So, like you said, it scales similarly, it scales down to zero, which is maybe something that your other sort of container platforms and tools don't do. So it serverless in that way. It scales up at creating multiple copies so your code needs to be stateless, right? So that's similar. You don't manage the infrastructure, so that's also common. I think the difference is, though, you do take on managing the run time. So you do take on saying, "Okay, I do want to specify my Node version. I'm now responsible for knowing what is the right Node version."
Which can also be good because maybe, again, you're like, please don't upgrade the version of Node without me approving it because I'm willing to take the risk that some performance enhancements come along or security patch happens, but I need to know that my app is not changing out from under me. So, again, that's important to some people. But the actual underlying infrastructure sort of like the networking and the creating of the nodes and the machines, that's all virtual machines, that's all transparent as well. Things like scaling down to zero, but then being able to scale new instances in milliseconds, really quickly, that's similar. You set timeout similarly. Like, oh, I want this to run for six minutes or whatever as a maximum, that's similar. You still have the issue of cold starts because you're creating those new instances. I think maybe containers can have some answers there, which we can get into as well. But a lot of it looks very similar to the functions as a service, just with the added thing of, again, you're also taking on managing that run time.
Jeremy: So let's talk about infrastructure for a second. So, again, maybe you want to block your kid's ears if they're in the car with you, but Google Cloud Run does run on Kubernetes, right? But you don't have to manage that. You don't have to manage the underlying Kubernetes when you're running this though, right?
Bret: So Cloud Run actually comes into different platforms. One is the fully managed, so this is where you are not aware of any of the actual underlying infrastructure. Hey, here's my container, go nuts. But there's also a version called Cloud Run for Anthos, which runs actually on Kubernetes. So the fully managed version is... I don't know how much we talk about what exactly used to run it, but it's not really Kubernetes that we run it on it. It's the same serving infrastructure actually that we use for Google Cloud functions. So this is not Kubernetes. The version called Cloud Run for Anthos, which we can talk about later, does actually run in Kubernetes. So the idea here is they have compatibility at the container contract level, which basically means your container needs to listen to traffic on a defined port.
That's basically it. There's a project called Knative, which is like this open source standard. So the fully-managed Cloud Run is completely compatible with the Cloud Run that runs on Kubernetes and Cloud Run for Anthos because of this Knative standards. So the APIs are similar, the container contracts are similar. The difference being version that runs on Kubernetes/Knative, you could potentially take Knative and you can run it wherever you want, like on premise or in your laptop or whatever. People do like that because even if the fully-managed version of Cloud Run is easiest to use and to set up, and it's probably the right answer for a lot of folks listening to this, because it's the most sort of serverless. You don't install, you have it, any knowledge of anything Kubernetes. But people want to know, okay, if Cloud Run ends up not being a great option or I have like really specific needs, I have to actually run this in my data center. The in between Cloud Run for Anthos and Knative, it gives you that open source portability and flexibility.
Jeremy: Great. So what about actually connecting to these things, too? So, with Lambda or Azure or even Google Cloud functions, you have events, and you have the ability to use API gateways or HTTPS connections. So what are your options for connecting to the applications running on Google Cloud Run?
Bret: So, right now, it's all over HTTP. So that means it's a good fit for sort of your external facing applications. With security controls, you can set it up so that, okay, this isn't actually open and accessible to the internet. Using HTTP, you can talk to our pub/sub system, which I guess that's similar as SQS or SNS. I forget. I always get those confused.
Jeremy: SNS is more like the pub/sub, yeah SQS is the queue.
Bret: Okay. Yeah. So like an SNS equivalent, where there's these events that get published internally within Google Cloud that can feed into pub/sub, and then that can then push into Cloud Run or push into whatever. Think of it as a fan out mechanism. You can set that up to be secured, but it still works over HTTP. So your function signature actually looks the same, whether you're responding to an event or responding to an HTTP request, the difference being, what does the payload look like that you get.
Jeremy: Cool. All right. So what about the developer experience? Because this is the other thing, too, where whether you're just uploading a snippet of code, obviously you need to upload a docker container that has your code packaged in it. So how do you do that with Cloud Run?
Bret: Yeah. So I think one other thing that is a little more of something to think about when you're developing for Cloud Run and containers is, yeah, you don't just say, "Hey, here's more my source code." Go nuts. You need to take this dockerfile, turn it into a container, which is basically, again, a compilation process. Then you need to publish that to some registry somewhere so that Cloud Run or whatever is that's running your container knows where to pull this from. It's kind of this you publish it. So we have a tool at Google Cloud called Google Container Registry, but there's also like Docker Hub and a bunch of these other repositories. So there's that additional step and really kind of maybe two additional steps of compile your container, you can turn it from a dockerfile into an actual image and then publishing that image.
Then when you deploy the Cloud Run, you say, "Okay, use this deployed image." So there's a couple of extra steps there. Now, one of the things you can do, like I'm an app developer, I'm not an Ops person, so I want to do as little of that as possible. So a couple of things you can do, one, you can just script it, right, because it's pretty much the same thing. It's like, okay, take my container, compile it, publish it. Docker is smart enough to know like, oh, all he did was change these couple of lines in the source code. I don't need to rebuild the whole container. I can stop, start from the previous step and just sort of redeploy the code into the container image, and so that will speed it up.
But there are also tools that actually will build these containers for you. So a lot of folks have probably used Heroku or Pivotal and a lot of... That platform, you can use the source code based, right? Like, take my code, hey Heroku, hey Pivotal, you build my container for you. Whatever you need to do, I don't care. So they actually collaborated and they created a project called buildpacks, which is now kind of a governed and open the CNCF, the Cloud Native Computing Foundation. So that's like an open source standard on all these buildpacks. One of the main ways that buildpacks are useful is to take source code and turn it into a compiled and potentially deployed container without you having to write a dockerfile or know anything about it.
Now, in full transparency, I've used these buildpacks a bit, they're still a work in progress. I think you still kind of have to know what it's expecting and structure your app appropriately. But it's open source. It's come a long way. You can create your own custom buildpacks so that, again, you as a developer don't have to know what a dockerfile looks like. It sort of will write and compile it for you. Then some programming languages have this built in. So Java has a project called Jib. Go has a project called Co. Because languages know the most about themselves, they can kind of more intelligently build and publish or build these containers for you. So I would say the developer experience is a little bit more to do than just, here's my source code and go. But I think that it's headed in the right direction because I think, again, as an app developer, I don't want to have to mess with.
But, again, from the Ops perspective, obviously, they like having that dockerfile, they like having that container because it's auditable, right? So if I say, hey, buildpack or Jib, whatever, go build this for me. It's useful, but God only knows what it actually is creating. It's probably fine. But if you're talking to your security team, they're going to say, "Well, what is he doing?" You're going to say, "Well, I don't know," but look how easy it is to use. So they like having that auditable dockerfile because they can go through line by line. It's installing this dependency. It's from this operating system. It's installing this security library. Oh, okay, that's an old version. We need to update that. Again, it's all about tradeoffs.
Jeremy: Sure.
Bret: I think one of the messages I want to kind of get across is that maybe moving to something like containers and Cloud Run, it feels like you're taking on way more than just getting started with functions as a service, right? Your Lambda, your Azure functions, or Cloud functions. And there is. There is a little bit more to manage. I don't think it's a huge amount of overhead, but I think what this really does is it enables a lot of your existing workloads to start to be serverless. Because we all have apps that have we started before serverless was a thing. Even if it's not perfect, we would love to get them to be serverless.
Jeremy: Yeah, totally makes sense. All right. So one of the things that, going back to the serverless containers thing, it does enable, like you said, different types of workloads that you can do. One of the things that's really, really interesting to me is the concurrency model here. With Lambda functions and with Azure functions, it's this sort of one concurrent connection is one instance. I think Google Cloud functions might have two sometimes randomly or something like that, but it is radically different with Cloud Run. So can you explain how concurrency works?
Bret: Yeah. So Cloud Run enables concurrency. So, if you don't want it, you can just set concurrency to one, and then it will behave exactly as you said with cloud functions or Lambda functions, whereas an instance only handles one request at a time. That obviously has a lot of advantages in terms of how you reason about it and simplicity because you just know if I have a 256 meg function and is handling request, I have 256 megs available to me right now with some caveats. But the problem is, and we see this with customers a lot actually, they get started on cloud functions. Everything is great, super easy to use. They put more and more workload, they put more and more traffic into their cloud functions, and eventually they come to us, and they're like, "Hey, this actually is starting to get quite expensive."
I think that's something that maybe I wasn't expecting or we don't talk a lot about because of the really big cost benefits you get from serverless. When it's very spiky workloads, you think like, oh, it's always cheaper. But actually, a lot of times, under a heavy workload, it can be expensive, especially to just raw compute or something, right? So one of the things like web frameworks have been good at and been working on for a long, long time is handling concurrency and really being able to handle a ton of requests from a limited number of instances. So Cloud Run lets you address that by saying, okay, instead of concurrency one, why don't I say I can say concurrency 10, 15, 30, 80, right? So that tells the Cloud Run, watch this container. You can send up to 30 or 40 or 50 requests to it, and Cloud Run will monitor the resource usage in that container so that if it sees, okay, it's busy, I probably don't want to send any more requests to it.
Even if it, even if it hasn't hit that threshold, then it'll kind of do the cold start thing and create a new instance and redirect traffic to it. But concurrency is nice because you actually can get massive savings in reduction in cold starts because in the ideal world, it goes when handling one request at a time to 80 requests at a time. That in theory is one cold start versus 80. Now, obviously, that is like a very academic that extreme. But even if you cut your cold start requests by a third, a fifth, right, that can be a really significant improvement. Especially if you're doing something like a web app, right, where it's user facing, and you really want to minimize those occasional cold starts because a user actually feels that pain, and you want to minimize that as much as possible.
So it has a couple of advantages with concurrency, one which is performance, so fewer instances created. Then the other thing is from the billing perspective because essentially, you pay as long as the instance is hot and serving traffic, rather than have one particular request, which means, again, if you're just reserving one request at a time, you're billing looks basically similar to your Lambdas and cloud functions of the world where it's just pay exactly as long as it's handling one request. But since you have concurrency, while it's handling that one request, if another request comes in, it handles, finishes it, maybe a third request comes in, handles it, finishes it all while that first request is happening, you actually pay the same.
So, as long as there is one request being handled, it doesn't matter how many other requests are also being handled, it's the same billing. So you actually can see pretty significant financial savings because you're taking advantage of, again, what these web frameworks have become really good at doing, which is handling concurrency within an instance. So that's nice. Now, it's not all for free in terms of developer experience and productivity because as a developer, you actually now have to be aware of, oh, I could have other requests come in while I'm handling this, so I don't have complete access to this instance.
So kind of the thing is we deal with around being thread safe and making sure I don't use too many resources because it'll impact other requests coming in. You have to be aware of that. Now, I think that's important, but I also think that we have years and years of experience. I would say it may not be a well understood problem, but is a well stack overflowed problem. So there are just tons of help around like, okay, here's how you can help solve these concurrency issues in your application. But again, if you don't want to mess with it, set your concurrency to one, and it will behave exactly like your cloud functions does now.
Jeremy: But I do think that it actually does give you a huge advantage. I mean, not just from the billing perspective, but also from the type of applications that you are building. I mean, if you're building something that needs to respond within 50 milliseconds or something even less than, as soon as one of these containers spins up, and you've got the concurrency set to 80, you immediately can handle 80 users. I'm assuming the system is probably smart enough that as the traffic continues to build up, that it probably starts spinning up new containers and kind of getting those sort of ready to go. But I think that is a much different approach. I mean, even with Lambda functions, you have this initial scaling boost, and then it scales, I think additional 500 concurrent invocations every five minutes or something like that. So you do have some of these scaling problems where you kind of have to warm things up beforehand. So I think that's really, really interesting way that that works. So what are some of these applications? Why would I maybe use Cloud Run versus just using regular fast beyond the billing side of things?
Bret: Yeah. We touched on this a little bit earlier, but I think part of it is really helps you with applications that were not designed as brand new Greenfield applications, once serverless and Lambda and cloud functions and Azure functions were established, right? So I've got this web app, and I would love to move it to serverless. But it's got a bunch of end points. I've got an API, right, and I've got a bunch of end points built into this API. It's got shared libraries within the code base. I guess, and I would love to be able to tease these out and just slowly move them to different functions if that's the way that I want to do that. But it already exists. It's useful. It's going to be real hard to convince management to change this.
But if I couldn't take that and move that serverlessly to a serverless environment platform, that would be great. Because now I get all the characteristics of no longer having to manage infrastructure, the scaling and building that we talked about. So I think for user facing web applications, for APIs, things like that that have a lot of you... I mean, I guess maybe we can use the word monolith. So there's monolithic applications. That really helps create this environment to run them in. So that's one thing. Then the other is just if you have applications that just aren't supported by your cloud providers, languages, or libraries and things, and you just want to be able to customize that runtime environment. So Cloud Run is there as well. Also if you need your application to be portable. So I want to run it in Cloud Run. Okay. Everything is working great, but now I also want to run it or some version of in my data center because I have these constraints, or I have these legal requirements, or I have hardware requirements needs, right?
So I need GPUs. I actually have to have eight cores and I need X amount of Ram. That's not provided by sort of the shapes that my provider gives you. Cloud Run and it's platforms gives you that flexibility. It actually opens up a whole big world of new workloads that you can run serverlessly that aren't just functions as a service. Then I guess the last thing would be is if you're already running stuff on Kubernetes and it's in containers and you don't have a lot of the complex sort of scheduling requirements may be that you need Kubernetes for, it's container based, right, just run it in Cloud Run. So it's a much, much easier way to get your containers actually up and running and serving traffic without having to manage necessarily a Kubernetes cluster.
Jeremy: Yeah. I think that's an interesting point, too, about things like GPUs or any sort of special hardware requirements because certainly that is something that can become a limitation or is a limitation on most platforms right now. So you still would have to run your own Kubernetes cluster if you wanted to have the benefits of things like GPUs. But you could basically have a very portable workload that you could move back and forth between sort of managed in your own clusters. So let's talk about this Knative thing because I think there's a lot of confusion with Kubernetes versus Knative and how work you have to do. Knative is the serverless fast component for Kubernetes or whatever. So just take a second, explain what Knative is just in case people don't know.
Bret: Yeah. So Knative is open source project. It was started by Google and some other partners like Red Hat and Pivotal and TriggerMesh and IBM, all contributing to basically put a layer on top of Kubernetes to show to a developer a more serverless environment. So, as a developer, if I'm deploying to a public cloud like AWS or Azure or Google, I don't care what Google's Ops team has to do, right? That's the whole sort of point of serverless. But there's a lot of use cases that you can talk about where you can't use public cloud or you can't use the particular platform that they offer you because there's some limitation. It would be nice if, as a developer, I could still deploy to this serverless tool. It happens to be managed by my Ops team behind the scenes.
But as a dev, it looks the same to me. I deploy it, it scales up, scales down, gives me all of those same characteristics that I'm used to, but my Ops team is doing that. So Kubernetes by itself, if you've ever written, you just quickly drown in the sea of YAML configuration files and really managing Kubernetes is hard. It is a full time job, at least. So you don't want to take that on directly, even as a developer. Even if someone's already set it up for you using it as a developer, especially as a serverless developer because we've sort of seen the Holy grail of like, yes, this is what we want the experience to look like. You still don't want to set up all these configuration files and create a deployment and a service and creating pods and all of this stuff.
So Knative is an attempt to put a layer on top of Kubernetes to start moving in that direction. You can use Knative directly. You can just deploy. There's a utility called KN. So you just can do a KN deploying and deploys into your Kubernetes cluster running Knative. It's a really nice experience. The philosophy behind Knative, though, was not actually to create an open source FaaS that runs on Kubernetes, it was actually to have these standardized building blocks for pieces that serverless platforms use that's not maybe built into Kubernetes. So, by way of example, I can't remember the last count, but let's say there's 20 plus open source FaaS platforms out there that run on Kubernetes because Kubernetes gives you all these nice benefits of scheduling and resiliency and all that.
But there's a bunch of stuff that everyone is just building again from scratch. So everyone is building a building component, where it goes from source code to deployable container. Everyone is building a serving component, which is okay. A request comes in, how do I route it, which version of my code is running, do I have multiple versions? What percentage of traffic? It's what? So everyone is sort of building that piece. Then like eventing, everyone is sort of building some eventing thing. So Knative was attempt to say, okay, let's just all build this together collaboratively. Then your value add as a serverless platform sits on top of that. So, by way of example, SAP, right, they wanted to give a serverless experience to their developers, but you can't say like, "Oh, Hey, SAP developer, just deploy AWS Lambda, deploy to Google Cloud, right? There's just no integration. They are already building a platform.
So they can build a platform on top of Knative that looks and feels like SAP and has all the idioms that SAP uses, and it feels very natural to an SAP developer. They didn't have to rewrite all the buildings serving in eventing piece for them. So, if you were going to build a platform as a service, a serverless one, Knative gives you a lot of really, really, really useful building blocks to build on top of. Again, you can use it directly. It's not really designed for that, but it's fine, and it will work. I think maybe the bigger takeaway if you're in the context of Cloud Run is... Cloud Run is compatible with Knative. So what that means is if I'm just sick of Google Cloud, I want out or maybe more realistically, management has shifted priorities and we're going on premise, whatever.
My app doesn't have to change much because Knative essentially gives me that escape hatch, right? So, if I'm worried about Vendor lock-in and that sort of a thing, I can take my containers, and I can run them on Knative in a very, very similar way to Cloud Run without having to rewrite and rearchitect the application. Then I guess even more concretely, I mentioned before, there's Cloud Run, which is fully managed and then there's Cloud Run for Anthos, which runs... Anthos is a whole other topic, but basically, it's kind of this suite of Google Cloud tools based around Kubernetes that you can run on premise, in Google Cloud and any other cloud. There's a whole bunch of things that you can sort of enable within it.
One of them is a Cloud Run. So, when you enable Cloud Run, it is running in a Kubernetes cluster on Knative. So it is actually running on Kubernetes on Knative, and then Cloud Run is just that little layer on top that's like the developer experience part just so it feels the same. But now once you're in Kubernetes, you or more likely your Ops or your DevOps team has access to all of the knobs and switches that Kubernetes has with again, GPUs and CPUs and custom networking and custom scaling. Never scaled below this number. I can add logic sort at almost any of these decision points where Kubernetes is managing your application. Someone in your team can do that for you. So yeah.
Jeremy: All right. I mean, that does sound like it's getting into the weeds quite a bit.
Bret: Yes.
Jeremy: So, for serverless developers, again, I think staying up at that Cloud Run level is probably where most of us would want to stay. So the other thing, and we haven't talked about this too much, but cloud functions, right, Google Cloud functions. So obviously they're different than Cloud Run because they're more traditional FaaS or fully managed FaaS. So what's sort of the difference between the Cloud Run and then just the normal Google Cloud functions?
Bret: Right. I would say especially for the serverless audience that's listening and maybe for the general developer, look at Cloud Functions, it has a lot of constraints as we all, just FaaS in general does. I don't mean that negatively. But if that works for your workload, it might still make sense actually just to start there. It is a simpler experience. Based on the philosophy of it, If nothing else, you're just going to have smaller pieces to manage. It's just easier to get started. So what we actually see a lot of customers do is starting Cloud Functions because the language they want is supported, they're not dealing with huge existing monoliths. Then if you start to grow out of it, either because your code base is getting too big, you're starting to put a monolith into this function or whatever or you start to need custom runtimes, hardware, fill in the blank.
Or it starts to get expensive, right? So I'm handling a lot of traffic, and I was experiencing too many cold starts, okay, now maybe you can look at going to Cloud Run. So if you aren't already using containers, I still think cloud functions is probably an easier way to get started. Probably, there's no reason you can't stay there. It works well. We have a ton of folks using it. It might be just sort of if you grow out of it, then you can move to Cloud Run. If you have needs that containers address, so again, your security team is like, "We need to be able to audit. What is your run time environment. We need to be able to control it with containers, or we need to be able to deploy your app to Cloud Run, Kubernetes, OnPrem. We want the benefits of those containers." Then obviously you would want to start with Cloud Run.
Then I would say the other difference currently is cloud functions has more, I guess, built in integration with the Google Cloud events. So things like our Firestore, serverless database. Cloud functions will natively integrate with, hey, whenever a database record has changed or inserted or updated, run a cloud function. So to do that with Cloud Run requires a little bit more work and also requires injecting this pub/sub service in the middle, which, again, is fine, and it works, it's just it's more work. So I think hopefully we'll have a little better eventing story around Cloud Run, but right now, if you're doing a lot of events, responding to stuff and events in GCP, in Google Cloud Platform, then cloud function is probably a better, a better fit, right? So responding to a file, getting uploaded or database change records and Firestore or something like that.
Jeremy: Awesome. All right. So let's talk about that then, the sort of the future of serverless in Google or Google Cloud because every major cloud provider, I mean, you've got Twilio building cloud functions, you've got Netlify, you've got ZEIT, you've got all of these different independent... There's like four new platforms and serverless development platforms that just launched I think in the last couple of weeks. It's kind of crazy where... Obviously the word serverless is way overused, and it's probably quite meaningless at this point. But I think, for some of us, we still believe in it. This is the future, right? This is sort of the way things are going to be built. This is the developer experience. I think a lot of people want this fully managed aspect of it, this idea of whether it's on demand, scaling and scaling down to zero or paying for value, however you want to say it. So what does the future of serverless look like at Google?
Bret: Yeah. So, if you'll allow me to have a little bit of hyperbole for a minute, I think... But I do like the philosophy that our team has around this, which is we love serverless, right? So I think the direction that Google has headed with serverless is we want to be able to serverless anything, run it anywhere and integrate with anything, right? So serverless anything, run it anywhere, integrate with anything is a pretty audacious goal. Realistically, will that ever happen? I don't know. I'm an engineer, so I'm allowed to actually be realistic here. But I love the philosophy of it. That sort of just multi-pronged approach of Cloud Run, which is super easy to manage, but then I have some requirements, right, that don't fit that.
Okay. Well, we still want to serveless you. Okay. So now we have Cloud Run for Anthos, and that runs on top of Kubernetes and Knative. So you can move it there and hopefully, yeah, it's a little bit less serverless or it's a lot less serverless for somebody because you're managing your Kubernetes infrastructure. But we still want to keep some part of it serverless for the developers. Then even if you aren't even using Cloud Run, not even on Anthos, on GCP or anywhere, okay, well, Knative is now this thing where you can still take those same workloads. It's open source. Kubernetes runs virtually anywhere, and you can move it. So I think being able to run on different clouds, different environments, different platforms, and then because it's containers, almost anything, you can turn into a serverless application by wrapping it in a container, at least as of now, assuming it's sort of stateless and some of those general caveats. Then integrate with anything.
So cloud events is in effort by the Cloud Native Computing Foundation to standardize what do event packages and payloads look like. That's built into the Knative spec from the beginning. So being able to respond to GitHub events and have your response running anywhere, right? GCP or otherwise, I think it's a nice goal, and I think we're making really good progress. Knative is developing really, really quickly. So I do like that philosophy again. Also if nothing else, if you're a serverless dev, and you're like, "Hey, I don't care about any of that," at least just know sort of Knative and those tools, that resonates well with your management, right? You can say, "Let's do this. Let's use Cloud Run. It's super easy to use."
But here's the thing. Boss or bosses' boss, we can leave, right? We can take this, and we can run Knative in our own data center or on whatever cloud, and our workloads don't have to change. I know that that appeals quite a bit to sort of the management types who are worried about lock in, whereas as a developer, maybe I don't care, right? I want the tool that lets me build as quickly as possible. Yeah. Yeah.
Jeremy: Right. Exactly. So, speaking about some of those tools as well, I mean, it's not all Kubernetes all the time or Google Cloud functions. There are a bunch of other sort of serverless tools that you have available. You have big query, you've got some of the ML engines that you have, your pub/sub you mentioned, cloud task, things like that. So maybe just say the suite of serverless offerings at Google. How do you kind of pitch that, I guess?
Bret: Yeah. There's a lot. I think starting back in 2008 before serverless was even a word, we had a project called, and we still have, it's massively successful called App Engine. App Engine, if you want to think of it, it sits in the middle between Cloud Run and cloud functions. Actually, I love App Engine. In a way, you can sort of think of it as it's like Cloud Run without having to do containers. So, if purely just the container part of it you hate, like Cloud Run. App Engine is nice because you can deploy an existing Node.js app in existing Python Flask app and it just scales serverless your web app with the same sort of... It has concurrency. It has same similar scaling characteristics of scaling down to zero. But you don't even have to mess with containers. So I love App Engine. It's probably one of my favorite products on Google Cloud.
So that's kind of the three pieces of compute for Google Cloud serverless. You mentioned databases. We have firestore, which is like a document-based data store, sort of like your MongoDB equivalent, but one thing that's nice there is it's intended to be directly integrated with or can be directly integrated and to talk to mobile and web apps, like the clients directly, talk to the database by design. again, part of serverless is productivity and only spending time on business values and how many of us have built an app. The whole freaking app is just some front end on API layer and a database. So you have the front end just talks to the API, API says, okay, this request looks good. It talks the database because data back returns it to the front end or whatever, right?
So there was a whole categories of applications now where that little API layer just disappears, and you just have the app talking directly to the database, which is scary for a lot of people, but it actually works really well. There's a whole suite of security rules and off and stuff like that built into it. So I'm a big, big fan. So firestorm came out of kind of I guess a marriage of a previous database we had at Google called Data store and then Firebase, real time database. So, Firebase, if you don't know, has a lot of developer tools for mobile and web apps. Love, love, love the whole team at Firebase and the products they put out. They had a real time database that was based on JSON, and so basically...
I know putting a lot of terms out there, but the main thing to take away here is Firestore, super high scalable, like a document database, serverless database DB. So database, it scales up and down. But you can also talk to it from clients, which means you can really simplify your development. It also does cool things like real time syncing, right? So one client chases some value, and within a second, all the other clients who are listening to it get an update and a notification. You can really simplify your development with Firestore and Firebase tools. What else did you say?
We have serverless tools for machine learnings. So, if you're doing Tensor flow or those sort of things, we've got several tools. It's called AI platform that scale up to do all of your training. This is a very spiky workload, right, with machine learning because you're like, "Hey, go analyze these 10 million images to learn how to solve some problem." So that's a lot of work to do upfront, but once you're done training, right, you want to scale that all back down to zero. Then now your serverless is the predictions or request comes in. "Hey, solve this problem for me." "Okay, cool. I'll solve it. Here's your response." That scale serverlessly just like incoming requests to a FaaS. So data, machine learning, big data, you mentioned BigQuery. So that was also kind of a serverless tool before sort of serverless became a popular term. You just write standards SQL and it can query just massive datasets, gigabytes, petabytes of data in just seconds or minutes.
It works very serverlessly. You don't have to provision any infrastructure, you don't have to manage anything. It just scales it all up behind the scenes. Some people have kind of even tricked it into being almost as like sneaky little FaaS because you can now write user-defined functions. So you could write little JavaScript function that runs on every record. So now it's starting to blur the line I guess between where's your FaaS in your database. So a lot of tools around big data, machine learning, databases and compute on Google Cloud.
Jeremy: That's awesome. All right. Well, listen, I mean, I think that Google has a ton of great stuff when it comes to the serverless things. I really am a big fan of this Cloud Run thing. I think for certain types of workloads and certainly if you are building things that need special hardware or even if you're just using the managed version of it, I think there's a lot of these workflows where this concurrency model could be really, really interesting. So, anyways, listen, Bret, thank you so much for being here. Obviously, you're always out there speaking. As you said in the beginning, people can contact you. So how would they do that?
Bret: Yeah. Best way is probably on Twitter. I am @bretmcg, B-R-E-T-M-C-G on Twitter. That's probably the best way to get ahold of me. So send me a message, I'd love to hear what you're doing. I also recently have started co-hosting a video series called the Serverless Toolbox. So, if you go to serverlesstoolbox.com, we have some episodes about doing serverless things, mostly around GCP. Our first episode, for example, I want to create a Rest API to serve out some data, but instead of being backed by a database, I actually want it to be backed by Google Sheets because it's just so easy to have anyone in your org set up permissions, go in there and just change data. So it's a very fun, kind of a little bit silly, lighthearted series. We've got props and costumes and things. So check that out at serverlesstoolbox.com or you can reach me @bretmcg on Twitter.
Jeremy: And if they want to find out more about just what Google has to offer from a serverless standpoint, it's cloud.google.com/serverless, right?
Bret: Yep. Yep. That is the landing page. Would love to hear your feedback.
Jeremy: Perfect. All right. I will get all that in the show notes. Thanks, again.
Bret: All right, thanks, Jeremy.