go podcast()

A follow-up episode on last week episode. We go a little bit deeper into Encore with André Eriksson. Encore can do a lot for your Go project and infrastructure. It allows your team to focus on your product and provides local development and DevOps tooling that help your team go faster.

Links:

How to support the show:

Creators & Guests

Host
Dominic St-Pierre
Go system builder - entrepreneur. I've been writing software systems since 2001. I love SaaS, building since 2008.
Guest
André Eriksson
Founder https://t.co/YGUYUS1Txv, the backend engine freeing developers from the maze of complexity.

What is go podcast()?

15 minutes news, tips, and tricks on the Go programming language.

Dominic:

Hey there. I'm Dominic Sainte Pierre and you're listening to Go Podcast. Today we are talking about Encore with the founder Andre Eriksen. Hello, gophers. So I have a special guest again this week.

Dominic:

So we are doing a follow-up on last week episode with Bill Kennedy. So turns out, that Encore is such a great product. That, yeah. I have Andre Erickson with me today. So thank you very much for accepting this invitation, Andre.

André:

Thanks very much for having me.

Dominic:

So Bill did a very good job, I think, to introduce Encore and, it piqued my interest a lot. And, I'm very glad that you are here today. But let's start with the famous, the startup question, if I can say that. So can you give us a little bit of background, about yourself, with a little bit bias towards Go and whatnot? So just so we know a little bit about you.

André:

Yeah. Sure. So I'm Andre, a long time software engineer. I'm the founder and CEO of Encore, but I've always been a software engineer. So before I founded Encore, I was a staff engineer at Spotify.

André:

And there was, Spotify at that time was mainly using Python and Java for back end development, but I really found that Go would be a perfect fit for Spotify. So I was actually spending a lot of my time building up the internal tooling at Spotify to support building back end services in Go. And that was everything from building sort of internal transport protocols. Spotify had something very similar to gRPC, but before gRPC, to sort of internal tooling and platform support, build systems, that sort of thing. So I spent a lot of time in the early days of Go just, you know, building up because Spotify has such a rich internal ecosystem of libraries that there needed to be a lot in place to be able to adopt Go as a language.

André:

So I was the the main person responsible for all of the platform tooling around Go at Spotify. It turned out that, I mean, even to this day, Go is not a very big language within Spotify. It's mainly used for Kubernetes tooling and and, back end services related to Kubernetes. But that's sort of how I got into Go. And I really fell in love with the language so much that eventually I just felt like I need to start a company that is really centered around back end development with Go.

André:

So that was sort of how Encore came to be.

Dominic:

Yeah. That that sounds very interesting, to be frank. I had a couple of guests that that worked at, at Cloud Faire. And each time I'm hearing those, you know, those bigger organization adopting Go, it's well, adopting, you know, Cloudflare is already built on Go and things like that. But you know, just what you said, it sounds very interesting.

Dominic:

So so what about Encore? So it it came the idea came there. So I guess that you had some difficulties or you you had some problems that that you wanted to be solved at that time. And, was that when you you had the idea? You know what?

Dominic:

Let's build something to, to solve those problems?

André:

Yeah. You know, Spotify has a huge platform team. And they spend a lot of time building internal tools to make back end development more productive. But I was finding, even with that massive investment, it was still very, very laborious building, you know, back ends and scalable systems to the point where a lot of engineering time wasn't really spent on, like, the business logic of what you were trying to build, the product. It was actually spent on just dealing with cloud infrastructure and dealing with, you know, these very mundane tasks like maintaining documentation, defining APIs, all the sort of boilerplate around that.

André:

And sort of every part of the organization was reinventing these things. And then when you start to look outside of Spotify, it turns out that this was a problem that not only Spotify has, but every company in the world is sort of going through exactly the same journey of, you know, okay, we feel the pain around sort of DevOps and platform engineering and internal tooling. So we're gonna set up a platform team, and then we're gonna work on Kubernetes automation, and just everyone is sort of doing the same things. And it turns out that that's both sort of a waste of time that every company is just repeating what every other company is doing. But it's also that you don't really get that far because there's so much work to do, that you end up just really scratching the surface on what a really amazing developer productivity and developer experience could be.

André:

So, I started Encore to really build something that's much more integrated and much more productive than what you typically see when you're just, you you know, we have a platform team, and they're working on sort of internal tooling and and Kubernetes automation, which is most of the time what you end up doing. So, Encore is a much more productive and integrated experience. And, that's sort of the main selling point I would say.

Dominic:

That really talks to me because I am someone that build products. So I've I've been in in the startup world for like 2 since 2008 and and building my own product. But let me let me do the devil's advocate a little bit here. So Yeah. If I'm understanding what Encore is trying to do exactly, so to whom exactly are are you what what is your target audience exactly for that?

Dominic:

Because if if I understand what you are saying, bigger organization already have staff to do to do all of this, you know, automation and all all of this things. So are you expecting them at some point to, to add up Encore? Or you are targeting maybe, you know, smaller smaller companies like maybe startups that that might be, I don't know, 10 or 15 people in there and they don't really have, you know, staff to do to do DevOps maybe?

André:

No, both. I think DevOps is really just a small part of it. If you look at, you know, there are a lot of larger companies that are looking at Encore because it's much more productive than anything they've built in house. And that is because, typically, when you're a big company, you have a lot of, you know, Spotify internal tooling is much worse than what you can get through adopting Encore. And the same is true in most large tech companies.

André:

And it's not because Encore is just, you know, out of this world, it's the best thing ever. It's solving a different problem, right? When you have a large company and you have a platform team, you really need to solve problems that are sort of, you need to tackle the sort of common denominator that all teams are facing. And so you end up focusing on very generic problems. Like, every team is using Kubernetes, so we're gonna build Kubernetes tooling.

André:

It's very difficult for them to prioritize, you know, here is a very integrated solution that not every team will be able to adopt. So what Encore is doing is we're saying, This is not for every product in the world. It's I mean, we only supported Go for a very long time. Mhmm. We are adding TypeScript support, and we are just shipping that.

André:

And we're gonna add more languages in the future. But it's very you know, it's a fairly opinionated set of languages, and it has an SDK that you use and so on. And so it's by being that integrated, you know, it combines the SDK that you use as a developer when you're writing code. It's combining that with the DevOps angle of sort of automating cloud infrastructure. And that means that that level of integration is not something that DevOps teams have the capacity to build because it's so integrated and so specific that typically what we see in larger companies is they have their hands full on much more generic problems.

André:

And so we see a lot of interest from both larger and smaller companies because the integrated solution, it's not for everyone, but if it is for you, it's, you know, an order of magnitude better than what you can find by sort of building it yourself.

Dominic:

Yeah. Yeah. I can see that totally. So I I did not have much time to, to look at it. But so far what I'm seeing and what I what I truly find interesting, maybe maybe I will say that, like that, is that, the flow the flow of the system that that is built automatically, the documentation as well, that is built, automatically when you start using the SDK like you like you are saying.

Dominic:

So is it is it is it the part of the static analysis that is done by by the tool exactly to to generate that? What what is the what kind of of static analysis is done and why, you know, why it's done on the source code?

André:

Yeah, so for those who don't know how Encore works, exactly like you say, Encore provides an SDK that you use when you're writing code. And that SDK has sort of built in, like, ways of defining, like, here's a database. Here's a Pub Sub topic. Here's a secret. Here's a cache.

André:

Here's some configuration. Here's a back end service. Here's a, an API endpoint, and so on. And it provides all these tools as sort of, they're the building blocks of modern back end development, right? Those are the things that you use to build a back end.

André:

And because Encore provides these things, there's like a standardized way of expressing them. You know, if you wanna define an API with Encore, there's a standardized way where you define it. And that way is, it's very simple if you're a Go developer. It's just a normal Go function in a package. And then you add an annotation, which is how you tell Encore that this particular function is an API endpoint.

André:

If you wanted to find a database, you import a package from Encore's Go SDK, and then you just define an object in your package, and that object represents your database. That's that's sort of how it works. And because of the standardization of, like, here's the way you define a database, what Encore does is it can then parse the source code. It's not running the source code, it's just parsing it. And it's pulling out all of these concepts from your source code.

André:

And it's building up a graph of your system. And it's building up this understanding of how everything fits together. So if you're a back end developer and you know, I'm interviewing you or I'm asking you questions and I say, hey, can I can you explain how your back end works? What most engineers would do is they would go to the whiteboard and they would draw boxes and arrows saying, you know, I have this service over here and then I have that database over here. And when you call this API, it writes some data to this database.

André:

And then it calls this other API and it does some more stuff. That's sort of how we, as engineers, think about systems. Right? And Encore, by just parsing your source code, it understands all of the same information because it's so standardized. And that's a really unique capability that, you know, it's not something we've seen in developer tools.

André:

This knowledge of how a system fits together, it's something that until Encore has really only lived in people's heads. And so Encore is taking that and it's making it possible for tools to understand that. And like you say, once you have tools that can understand all of this information, then you can start to build really powerful tooling around that information. Things like rendering API documentation, rendering architecture diagrams, automatically provisioning infrastructure, and so on. So those are sort of the 2 parts to Encore, is you have this SDK that provides a standardized way of defining back end concepts.

André:

And then once you have that, then you have the ability to understand how everything works. And then you can just build really powerful automation on top of that. And that's the second half of Encore, if you will.

Dominic:

Yeah, that is very interesting. Am I wrong in saying that this part, the static analysis, is written in Rust, actually?

André:

We, for Go, it's all written in Go. All of the parsing and the static analysis for Go is all in Go. All the code generation for Go is written in Go. For Encore for Go, it's basically a 100% go. We've just launched a Encore for TypeScript.

André:

And there, most of it is actually written in Rust. And that's because there's really, really good tooling for parsing TypeScript in Rust. And so we've also built a a runtime for that, which is also written in Rust. So, but for Go, it's all, it's pretty much a 100% written in Go.

Dominic:

Nice. Interesting. Yeah, that's great. So so let, you know, let me, let me put the shoes of a of a developer that that would start to to give a shot at at Encore. So let's talk a little bit about the database because I, you know, I think it's, it's still it's still something that that intrigued me a little bit.

Dominic:

So by providing a a package yourself, so what kind of added feature that that we are getting as a developer instead of using the, you know, the typical database SQL package?

André:

So Encore isn't really trying to innovate on that side. What Encore does is it provides an SDK that has a standardized way of defining a database, right? So if you don't use Encore, the typical way that you define a database is, it's pretty laborious, but essentially you have a different setup for every type of environment, right? So when you're running locally you would use Docker Compose. And you would say, I'm gonna write this Docker Compose file, and I'm gonna create this Docker container, and it's gonna have the database, And then I'm gonna create a local password, and then I'm gonna have my database migrations, and I'm gonna have to write tooling to migrate the database when I'm running locally.

André:

And then I need to go into the my code base, and then I need to receive the the connection string, and then I need to create a connection pool, and I need to connect to the database, and then I need to and then for my testing environment, if I'm building a back end and I'm deploying it somewhere, then I need to create a different database, and it needs to run somewhere else, maybe in your cloud provider. And then you need a different password for that, and then you need to have a build system and a deployment system to set that up. And then for your production environment, it's gonna be different again. And maybe you use Terraform to set that up, and then you need a deployment process to migrate, And then you need a different connection string. And the list goes on and on and on.

André:

Right? And we haven't even gotten to the point where we actually use the database. We're just talking about the tooling to be able to have a database. So all of this stuff that I described, that's what Encore is replacing. So the difference is that with Encore, you import the Encore SDK and in your program you just say, you define a variable in your program, in your sort of service package, and you just say var my database equals, and then you import the go the Encore SDK, and you say, Encore dot new database, basically.

André:

And then you pass in, you know, the name of the database and some configuration, like, here are my migration files. And then Encore takes care of all of this stuff that I talked about. It takes care of setting it up on my local computer. It sets it up for unit testing and integration testing when I'm running tests. It automatically migrates the database.

André:

It handles connection strings and connection pooling and database credentials. It handles setting up sort of different admin users from read only users. It handles setting up a preview environment for every pull request. And then it handles setting it up for production environments and testing environments. So so it it it's it's really automating that complexity.

André:

And then this object that you get back, so you asked about the the database SQL package in the Go center library. The object that you get from Encore, you can actually get the standard library database object from that. And if you want, you can also get the pgx connection pool, which is the the database driver that that Encore uses under the hood. So it's not trying to replace the sort of standard integrations in Go. It's just trying to automate the complexity around using infrastructure and and setting up all of that infrastructure, if that makes sense.

Dominic:

Got it. Yeah. Yeah. That's, that's pretty interesting. What was was the design like that since the beginning?

Dominic:

When when you had this idea, did you already knew that you would have some some kind of of package that would, you know, wrap those, let's call that cloud, cloud services, like the cache, like the the database, and things like that. Was that the initial idea?

André:

Yeah. I mean, that was, you know, I think the original idea behind Encore, it came when I was at Spotify and we had just been spending a bunch of time building a new back end service. And, you know, at this point, you know, I've been writing, building back ends for over 10 years. At this point, I'd been at Spotify for 5, 6 years. I was one of the most senior engineers at the company.

André:

I'd been working with Google Cloud, you know, every day for several years. And we essentially had a production incident that lasted for several hours because we could not figure out how to get the permissions working correctly with a PubSub topic. It just everything was working fine when it was running locally because it wasn't using actual cloud infrastructure, a lot of it was mocked. It was working fine in the testing environment. And then for production it was just saying, you know, permission denied.

André:

And the cloud is so incredibly complicated, and setting up this, there's so many moving parts that it led to a production outage. But it was also just something that it was a massive time sink. And it was a whole team of people who are extremely experienced. And it's so incredibly frustrating that that's sort of the best we can do as an industry is that we don't have anything that's sort of more optimized in terms of workflow than spending days or even weeks, you know, dealing with something as mundane as making sure you have permission to use something. It's pretty remarkable.

André:

And so Encore, from the start, it was really the way I realized that this static analysis angle is so important is because when Encore, you know, achieves this level of understanding that we talked about, where we have this graph, and that includes things like which services are publishing messages to which PubSub topics. And once you have that understanding, you can actually automatically manage permissions as well. So Encore will automatically, when it's setting up cloud infrastructure, it will automatically optimize and lock things down as much as you possibly can in terms of the permissions that every service should have to the point where you can say, this service has access to publishing to this topic because, and then Encore can point you at the specific lines of code where this topic is being used by that service, and it was approved in a pull request by this person, and it was then provisioned by Encore in this deploy, and the proposed changes were approved by this person. So you get this really, really auditable workflow, and the source of truth is the source code itself. But it's all automated, right?

André:

So you never have to spend any time and you never have any production outages because you don't, you didn't get the permissions right and you didn't catch that before production, right? So, it really is sort of security and productivity at the same time, which I think is fairly unique when it comes to this. It's enabled by this unique understanding that you get from the static analysis.

Dominic:

Yeah. There's a lot to unpack in what you just said. First of all, I really, really love those stories that we all have at some point when there's something that like that that happens, about the permission that you have, the problem. So, yes, let let me let me let me do this devil advocate a little bit again. So I assume that all of those information are available on on the, the Encore dashboard, I would say.

Dominic:

I would guess. Once you once you have Yeah. Is you know, do you see do you see some, how can I say that? So is is there, is there any any problem at the moment for for teams to to say, you know what? We will add up Encore, or Or is there any reticence that says something like this is a lot of trust that a company would put into Encore?

André:

I think, I mean, it's certainly different, right? I'm not gonna you can't really argue against that. What a lot of people, what's all of the stuff I've talked about, like all of the parsing and the static analysis, all of these things are open source. So you don't, of course, you have to have some degree of trust that the product works as you expect, as you want. There's, all of that is entirely open source.

André:

And what's powerful about that is all of this static analysis, you can actually run it, I mean, when you're running it on your computer, it's actually using exactly the same static analysis, which means that, you know, if you you're not gonna run into the issue of, like, oh, it actually made a mistake because it wouldn't run locally, if that were the case. The tests wouldn't work if the static analysis wasn't correct. But because it's all open source, you know, there isn't so much trust that you have to put into Encore the company. And the proof in the sort of the maturity of the product, it's quite evident once you actually use it. I've been working on this for getting close to 10 years by now.

André:

And it's really quite a mature product. And yeah, so I'll stop there. We can sort of go in whatever direction. But that's sort of the key is you don't have to trust us as a company because all of this stuff is open source. And you can build your own tooling around it.

André:

We have banks being built with Encore, and they are using the stack analysis themselves to sort of power internal tooling on top of that. So, there's a lot you can do to sort of leverage this tooling yourself as well.

Dominic:

Sure, sure. It's pretty exciting. I mean, don't get me wrong here. I'm just trying to I'm just trying to say what what what I feel not having used it so far. And this is also some feedback I got for my own open source project.

Dominic:

At some point when you propose some back end solution, there's always some I don't know. Well first of all, it's pretty hard to to have a team, migrate to to something new. So maybe, maybe maybe with the maturity that Encore is adding, it's not not an issue at all. So, you know, I'm just, I'm just someone that does not really know the the products of might might be

André:

might be a bad question. No, I think it's a very real problem to be sure. I mean, let's be honest, people, not everyone likes change, for 1. But for 2, what we've seen to be very successful is, you don't have Like, encore is not an all or nothing proposition. It you can both sort of take it and you cannot use the infrastructure automation side.

André:

You can just use the open source version. There's built in support for building your own Docker images. And then you can just take those Docker images and you can deploy them to your own existing environment. And that's like an easier way to, if you don't wanna change the sort of cloud infrastructure aspect, for example. But another way of doing it is Encore can integrate with your existing infrastructure.

André:

So particularly if you're using Kubernetes, you can just point encore at your existing Kubernetes cluster, and then Encore is not going to go off and create anything outside of that. It's just going to manage your backend services in its own namespace inside Kubernetes. And then if you then go and define additional infrastructure, Encore can create that. But in that way, you can sort of, you can experiment with Encore only for, say, one service. And either by building a new service or converting an existing service.

André:

But that means that it's, you don't have to sort of make the decision for the whole company. You can just try it in a team and see, like, is this right for us? You don't have to make it into such a big decision, really.

Dominic:

Oh, yeah. I like that a lot. I mean, this, this this might smooth the, the adoption for sure. You know what? Let's return a little bit, you know, regarding the, the endpoint and the, the services itself.

Dominic:

So I I really like what what Bill was, was saying about, you know, at the end of the day your your endpoints become, you know, Go functions that you accept types basically and you return types. And instead of instead of working with the protocol itself with the HTTP response and request. So, you know, that that is fantastic. My my question though, because I I was I was looking around a little bit. So what about what about WebSocket, for example?

André:

Yeah. So Encore, basically, the way we think about it is, the so we call this the type safe API endpoints. And those are the endpoints that you define as normal Go functions that accept types and return types. And what's so powerful about that is, well, you don't have to deal with the boilerplate of parsing the incoming request or serializing the response. But also when you're making API calls between services, you also get type safety in the API calls.

André:

So you never have to worry about, like, am I passing in the data correctly and so on. But you're absolutely right. This level of abstraction is not always the right one. So we think of it as sort of escape hatches. And we make sure that for all of this functionality there should be and there are escape hatches where you can sort of drop down to a lower level of abstraction.

André:

So in the context of API endpoints, you have the ability to define what's called a raw endpoint. And what happens then is it's just a normal Go net HTTP handler. And then you do get access to the HTTP request. And that's what you use when you wanna do things like WebSockets. We're gonna add support in the future for having, like, type safe streaming requests and responses, so that you don't have to use WebSockets if the thing you wanna do is really just like a streaming data back and forth.

André:

But raw endpoints work great today, and there are a bunch of other use cases where you might want them, whether it's things like accepting webhooks. Oftentimes then you need to have more control over the request body because you need to parse it in a certain way and do some validation and some verifying signatures and that sort of thing. So you always have that escape hatch if you wanna do something where Encore's abstraction doesn't really make sense.

Dominic:

So the the typesafe an handler, is is it is it required to have some, some JavaScript in there? Can can it can it be can it be configured to accept, I don't know, XML, for example, or or it's only JSON?

André:

The typesafe handlers currently, it's only JSON. Probably, we will extend it to other sort of encodings in the future. Beside, I mean, XML these days seems like a niche use case. And I think we're fine with, you know, if you wanna do something that's very niche, we don't need the type safe endpoints to support that because you always have the raw endpoints that you can drop down to. So we would probably say you can use the raw endpoint for that.

André:

But, something like protobuf, we wanna support with a a type safe endpoint. And then we can either support it as, like, HTTP and JSON, or GRPC. Mhmm. But I think, in a lot of ways, I think what Encore is advocating for is more the philosophy of, like, it's more productive to think in terms of data structures and in terms of semantics. Like, when you define an API, you should really be thinking of, like, what promises am I making to the caller?

André:

What are the sort of things that the caller must do before calling this, the sort of preconditions, and what are the outcomes that's gonna happen, the post conditions, and, like, what is the shape of the data that's being passed around? And those are things that it doesn't really matter, like, a well designed API from that perspective, it's irrelevant whether you're sending that over HTTP 1, HTTP 2, HTTP 3, gRPC, over, like, carrier pigeon, you know, it doesn't matter, it shouldn't matter. There are a very limited set of cases where the performance might matter, and you can argue, like, gRPC is more performant than HTTP and JSON. So, well, sure. But that's a different criteria than, like, okay, most people seem to care more about the transport protocol from the perspective of that's how you think about APIs.

André:

Whereas Encore is advocating for it's much more useful to define and reason about APIs in terms of the data structures.

Dominic:

Right. Yeah. Yeah. This this is, I I have to I have to admit that it's it seems sexy. Bill, Bill was trying to explain to me on, you know, last week about authentication.

Dominic:

And at the time, I think I understood what he meant. I'm not sure I I do these days. So let let's continue with this example. Let's say we have a service and we have, we have a type safe endpoint that I don't know. Just accept one type.

Dominic:

Whatever whatever what it is. But the user needs to be authenticated. What is it part of the precondition that you, that that you were mentioning?

André:

Yeah. So, Encore has essentially, for every API, you can say, there are 3 access levels. You can say, this is a public API that you can call without authentication. This is a public API that you have to authenticate to call. And the third is this is a private API endpoint, and that's just for sort of internal service to service communication.

André:

And if you define an API endpoint that requires authentication, then Encore will ensure that this function will never be called without valid authentication data. And what we mean by valid authentication data is you have to define inside of your system, you have to define an authentication handler, which is also just a normal function, and that handler will be called before your API endpoint. And if that handler returns, a successful response, meaning it returns, like, a user ID and some authentication data and a nil error, then the request is considered authenticated, and the endpoint will be called. And then Encore will actually keep track of the the fact that this authentication data exists. So if if that API endpoint goes off and calls another endpoint, Encore will automatically pass along the authentication information between APIs so that the other endpoint can also know that, okay, this user is the one that the request originally was from.

André:

And so that is automatically propagated by Encore between services.

Dominic:

Okay. So that yeah. I think that's the pieces I was missing. So let me let me try to to make sure I understand this time. So by adding a kind of a public or authenticated or private to your annotation on your function, this tells this this tells the system that it it will call some special handler called authentication or what whatever the name, but it it will automatically call that.

Dominic:

And when you say that if there is a second call, it will pass the the authentication value. So how how how would would it do that? If it if it's a type safe function, I'm just receiving the type that I'm supposed to receive. So where where will be that user information, that authentication value?

André:

Yeah. So this is part of Encore's, runtime. So Encore does a lot of things to keep track of the requests that are being processed. So when a function gets called, Encore does a lot of work behind the scenes before that, obviously. Things like it has an HTTP server, it's parsing the incoming request, it's doing validation, it's checking if it's authenticated by calling the authentication handler, and so on.

André:

But then it does things like distributed tracing, and request logging, and all this stuff. And all of this functionality lives in what we call the Encore runtime. And that's, I mean, it's really just like a Go package that we've written. It's all open source, so you can go take a look at it. But that package, it keeps track of, you know, which Go routines are processing what request.

André:

And it's quite similar to the context package, but it's actually associated with the goroutine itself, not with the context object. And that's how encore knows, you know, this, goroutine is processing this request. So Encore then provides there are built in APIs that you can query to access things like give me more information about the current request beyond the the type safe data structure. So you can access things like which endpoint is being called, what's the trace ID, what are the, you know, information about the payload, about the authenticated user? All of this information is available there.

André:

So that's how Encore keeps track of it. And then when you're making an API call, that's what's being used to sort of propagate that information to other APIs.

Dominic:

Got it. Got it. That that makes sense. One one thing I was, I was wondering is that I was surprised to see that there's no, you know, file storage in in the offering. So there's caching, there's Pub Sub, but there there's no, you know, blob storage.

André:

Yep.

Dominic:

Is that is that something that, is requested a little bit or or not at all?

André:

It's it's something requested. It's, the number 2, number 2, top item on our road map. So, it's coming quite soon. I mean, it's important to know that encore, it's not preventing you from doing anything yourself, right? Like, at the end of the day, it's just normal Go code.

André:

So, a lot of Encore users, they're already using Blob storage. And Encore isn't stopping you from using that in any way. It just, you have to do it yourself. Like, Encore isn't automating that part just yet. But there's absolutely nothing stopping you from, like, importing the SDK of your choice, whether it's Google Cloud Storage or AWS S3 or Azure Blob storage or whatever, it's, you know, it's not more or less difficult than if you weren't using Encore.

André:

It's exactly the same. If anything, Encore will handle everything else, which means that you have to sort of do the last mile of setting up a cloud storage bucket and defining the permissions. Encore provides some Terraform providers to make that easier. But at the end of the day, there's nothing stopping you from using that or any other cloud infrastructure if you want. But I completely agree with you, that's probably one of the more common things that we haven't added support for, just yet.

André:

But it's coming.

Dominic:

Yeah. Yeah. It you know, I I I fully understand that you can do that. It's just that the the promise of of of Encore is so great, at least at least to me so far. I I really need to try try it and and and whatnot.

Dominic:

But, I can see myself wanting to to use, abstraction maybe for the for the first time in a long time, I would say. So so that that that's just that's just the point. I mean, at some point being cloud agnostic is is kind of very, very interesting.

André:

Yeah. I mean, we like, Encore definitely tries to paper over or, like, make sure that if you it makes it really easy to move between clouds, for example. It's not I don't Well, my personal stance is, it's very difficult to be completely cloud agnostic. So Encore isn't really trying to be cloud agnostic. Because the difficulty is, if you try to be completely cloud agnostic, what happens is you have to sort of settle for the, the common ground between cloud providers.

André:

And oftentimes, you might choose a specific cloud provider because it offers something that the other ones don't have. And so it's actually quite important to me and to Encore that Encore doesn't prevent you from using the parts of the cloud that are unique to that cloud. And so, in the case of PubSub, for example, there are quite a lot of differences between PubSub behavior between the different clouds. And so Encore allows you to configure all of those things, but it makes it obvious to you that like, hey, if you configure this setting, that will not be portable. But everything else will be.

André:

So you can sort of decide for yourself, like, do I want this less portable option? Or should we go with something that's maybe we have to take on more of the complexity on the application side, but the the upside is that we, our application becomes more sort of cloud agnostic. And so we try to give you that flexibility and the ability to decide where on that spectrum you wanna fall, rather than taking that flexibility away from you, If that makes sense.

Dominic:

Yeah. Totally. Totally. I'm a little bit curious about, you know, what happens with with those, HTTP handlers that that that we are defining under our services. How are they deploy to the cloud?

Dominic:

Are they, you know, are you using like Cloud function under duet? Or it's really, you know, I I don't know. You know, what what what happened? Because if we return to my to my question, it's very nice to have access to, to a raw option to to enable WebSocket, but if I don't really know how how the handlers is going to be deployed, I don't I don't even know if if the connection is is even possible to, to to stay alive.

André:

Yeah. I mean, Encore isn't really trying to hide the infrastructure from you. It's just trying to automate it for you. So exactly how it is deployed is, in fact, it's entirely up to you. So when you use Encore's cloud platform, you integrate with your own cloud account.

André:

And then when you create, say, a production environment, and let's say you pick Google Cloud, then you actually have a bunch of options. Like, how do I want this to be set up? Do I wanna use Kubernetes? Well, do I wanna use autopilot? Or do I wanna use the sort of standard?

André:

Do I want all services to be combined into a single container for sort of small scale, Or do I want all services to be deployed individually in different Kubernetes deployments? Do I wanna use Cloud Run? And then if you use Cloud Run, do you want it to be scaling down to 0, or do you want it to be always running? Do you want CPU to be allocated even when there aren't any requests going on? All of these options, Encore makes available to you so that you can decide, like, what makes, what's appropriate for your use case.

André:

But then once you've made those options, Encore will fully automate everything else. So we try to sort of give you the knobs that you care about so that you can customize accordingly, But And then just completely automate the rest. So, for example, you can decide to say, actually, I want to use Cloud Run and I want to scale down to 0, then Encore understands what that means and will automatically configure PubSub to be push based. Because when when you have something that scales down to 0, you won't have any instance that can actually pull for new messages. So Encore will automatically make it a push based subscription.

André:

But then if you change your mind and say, actually, I'm gonna have this always running, then Encore can reconfigure it to be a pull based subscription because that's more efficient. And then if you use Kubernetes, it will be pull based by default. So it's sort of giving you the options and the knobs that you care about, but then it's making smart decisions everywhere else based on those decisions to sort of optimize and do the right thing, if that makes sense.

Dominic:

And you configure those option in in the UI? Or is it is it also available on the on the CLI when you are deploying?

André:

Currently, it's, in the UI when you create an environment. And then there's, like, an infrastructure page where you can you can modify it. But, yeah, UI based at the moment.

Dominic:

Sure. Sure. That sounds, that sounds very interesting. I mean, is there, you know, is there anything that that you would like people to know about that I might not touch on? You know, some, you know, some last, last thought on Alcor or anything else?

André:

I mean, I think we've covered a lot of it. I think, one thing that I really thought Bill touched on really well was just this notion of, I think when you look at the Go community, there's this, and I fully support this, but there's this really strong ethos of like, you know, doing everything from the ground up. And I think, to me, it's really powerful to know, and it's really, I mean, I love Go for the fact that all of this is available to you at your fingertips. But I think to me, it's really important to know, like, sometimes you're building something that's incredibly low level, and whether that's the next Docker or the next Kubernetes, sometimes if you're building something that's more high level, I think it makes sense to actually operate correspond correspondingly at a higher level of abstraction. And so, sometimes when I talk to gophers, they find that encore is actually too high a level of abstraction.

André:

And I think that sort of interplay is really interesting. And I think sometimes I feel like there's this maybe misguided perception that Go is all about just like only using the standard library. And a lot of time I talk to people and they almost feel afraid to suggest, you know, using anything beyond that. And I think that is a bit maybe misguided. Like, I think you have to sort of use the right tool for the job.

André:

And, this is not really a point about Encore per se, but more to say that I think the Go as a language and Go as a community, it's so much larger these days that we need to also be accepting of the fact that, like, different circumstances call for different solutions, if that makes sense.

Dominic:

Yeah, I tend to agree. I think though that the important part to me is, you know, for developers is to make sure that they understand what they are doing. And I think this might be where the suggestion or or of using the standard library is is coming from potentially to to make sure that, you know, just just make sure that you understand what what's the middle where, you know, how what what is the the lifestyle the life, time of their their their request response and things like that. That's that's the thing. But when you when you want to be productive and and you you are, you know, familiar with those things, I mean, that makes sense for sure.

Dominic:

Because if you think if you think about that, it's it's a little bit what Encore is trying to do as well. You know, if you want to go manual, then then you you you can use you can use so many so many tools that that you will be writing a lot of, configuration code yourself and and things like that. You can even go bare metal as well. Yeah. So yeah.

Dominic:

At some point, it's a it's a matter of choice and depending what what you're doing. As long as as people continue to understand what HTTP is, I think there's no problem.

André:

No, exactly. And I think, I mean, I feel like every gopher goes through a journey. You're learning the language, and then you start to use channels everywhere. And then you realize, oh, maybe I should pare back a bit, and maybe this was a bit crazy, overusing concurrency and that sort of thing. So, I definitely agree that it is a learning process and there's a lot of value in learning everything from the ground up.

André:

And I'm not trying to advocate against that, but different situations call for different solutions, I guess.

Dominic:

Yeah. Because if I, you know, if I'm honest, what what I'm really, really liking about Encore so far, to me as as well, at at this moment, is that I can I can see I can see myself finally building maybe an event driven system without having to to mess too much around with with the cloud infrastructure and whatnot? Which I think, if I'm completely honest, I think we went a little bit too far with I I I will say I will say it again but it it will be a quote quoted this time but sometimes we tend to do a little bit, over engineering, I think. And having to support all of this is kind of crazy when you think about that. So this tool this tool seems seems very nice.

Dominic:

I I I just can't believe how much complexity you are you are having to, to handle. Because I think I think you are also letting people, go manually configure things on on their cloud infrastructure themselves and things like that. And and the, you know, the the tool itself or at least the deployment part will will accept that and just merge the changes and things like that. I mean, this sounds so complicated.

André:

Yeah, no, that's correct. You can definitely do that. But I think you're right. It's definitely the case that you can sort of over engineer pretty much anything. And I think Encore, it's definitely not sort of advocating for massive infrastructure sprawl, right?

André:

Like, I think it makes sense to keep things simple. But I think with the cloud, this example I gave earlier with just setting up a database alone, it's so much work, right? Especially if you wanna do it in a way that's secure and locked down, and you wanna have different passwords for different environments, and different users for, different sort of, like, we we wanna have a read only user for, you know, business analysts to be able to query some insights about, you know, stuff that's happening. And then we don't want the production service to have admin access to the database because that's really dangerous. And the fact that just doing that alone is just an incredible amount of work to get right.

André:

And if you screw it up, you're gonna have a production outage on your hands. I think that's the part that really gets me is just software engineering by itself, even if you don't overengineer it, it's so incredibly complicated. And there's so much complexity that what I really, like, as a software engineer, what I really like, what I really feel strongly that Encore can add is the ability to bring that complexity back to like the realm of sanity, right? Like, why if you just wanna have a simple REST API that stores some data in a database, why does that take more than 5 minutes? Like, it doesn't seem like it needs to take more than 5 minutes.

André:

But that's sort of the reality. And I think that's really the problem, right? It's like simple products should be simple to manage. And as your product grows in complexity, so too should the tooling. But the reality of the cloud is just from the first minute, it's just insane complexity.

André:

And that's where I see Encore is bringing that complexity back into, like, reasonable levels where the complexity scales with your product, not with the overall complexity of the cloud.

Dominic:

Yeah. Totally. The the I mean, this only only this this is major. This this is exactly a you know, this is crazy when you think about that because now you you can you can finally start small. And when when you say complexity in your product, I would even add you know, when you start scaling in terms of users as well.

Dominic:

Because it. Yeah. It's not everyone that takes funding and and and, you know, start, having some some free user and things like that. That scale automatically to to 10000 users, you know? They they they're I'm coming from a a bootstrap world where, you know, we like to keep things calm and we like to grow slowly in.

Dominic:

But that also means that the the infrastructure that you are deciding at first will not scale at some point. And this is what I really, really like about Encore is that it seems to to to be capable to grow with me. So if I If I finally build something that people will want, I I know that Encore would would be there for me. So that is, you know, that is a great, a great offering.

André:

Yeah. I mean, I talk to a lot of startups, obviously, and a lot of them, they sort of feel that because they have ambitions to grow, that they need to start with Kubernetes. Yeah. Because otherwise, like, they would have to redo things later. And I think that's just asking for a lot of complexity for no reason.

André:

But I sort of understand it because if you have to spend, like, several months, like, re architecting things, you would rather just, like, start with Kubernetes, right? It sort of makes sense. But as you say, like, one of the benefits of Encore is you can actually just build your system and then deploy it using something that is really easy to manage when you're getting started. I mean, I really like Cloud Run for that use case. But then if it takes off, you know, you can deploy it to Kubernetes with the click of a button.

André:

Like, you don't have to re architect anything. And Encore can take exactly the same application and just set it up in a radically different way for a use case where you really want that scalability. And that I think is, as you pointed out, it's a real benefit because it means you don't have to sort of overthink it from day 1. You can just pick the infrastructure setup that works best for you today, and then Encore can change it for you in the future, if you need to. I also think, like, computers and Go, and, they're so fast that I think you can go much longer on something like Cloud Run than you would ever expect.

André:

And so I always encourage people to just start there, start simple.

Dominic:

Absolutely. And, you know, not only the infrastructure, but also it you know, you you start writing your application like it's ready to scale. But even even without to to have to to think about service to service communication and and all of all of these these things that at first are just let's be honest, you know, they are just distraction at first. You're trying to build a product, you're trying to solve a problem, you're trying to reach users. So that is also, you know, the second biggest reason for me.

Dominic:

That I kind of, kind of thinking I'm falling in love with with Encore without having started to to write a line of code yet. But I I will. And, I I yeah. I think I I've, I I cannot take you take you that that long again. But if if you're if you're, fine with it, I would still like to do a second episode in a couple months.

Dominic:

I don't know. 3, 4 months. When I when I I launch something, on on Encore. And and maybe I will have a new set of questions and whatnot. Just just just do a quick follow-up if if you are Yeah.

André:

Absolutely. Let's let's do it.

Dominic:

So, so again, we will have all the links in the show notes. So, again, I I really, really encourage you all to check it out. Like like Bill said last last week, I'm I'm also a little bit like that. When I see something new, I'm like, I don't know. But I assure you, you know, start to start to read the documentation, start to test it, and it seems to be very great.

Dominic:

Thank you. Thank you very much, Andre, for your time. I mean, this this was, this was great.

André:

Yeah. Real pleasure to be here. Thank you so much.

Dominic:

Alright. See you next week. Alright. That's it for this week. I would really appreciate if you can talk or share about this podcast.

Dominic:

It's it's always helpful. Also, another way to support this show is by purchasing my course. There's always a link in the show notes. So on that, see you next week.