Subscribe
Share
Share
Embed
Ev Kontsevoy, Co-Founder and CEO of Teleport, is back a year later to give us an update on what Teleport has been up to! Last time they talked the security growing pains of the pandemic were still a little manic, Ev is here to dial us in on what exactly has changed. Teleport is working hard to stay at the forefront of the “scaling stress” of cloud computing over the past year.
Ev goes into details about how Teleport has handled security and the massive shift to the cloud, and the four components of what access actually means and teases out their differences. Ev also goes into how Teleport works with authentication and identity and the innovation they bring to that space. Ev, and by extension Teleport, have a lot to offer. Check out his conversation with Corey for all the details!
Screaming in the Cloud with Corey Quinn features conversations with domain experts in the world of Cloud Computing. Topics discussed include AWS, GCP, Azure, Oracle Cloud, and the "why" behind how businesses are coming to think about the Cloud.
Transcript
Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: This episode is sponsored in part by our friends at VMware. Let’s be honest—the past year has been far from easy. Due to, well, everything. It caused us to rush cloud migrations and digital transformation, which of course means long hours refactoring your apps, surprises on your cloud bill, misconfigurations and headache for everyone trying manage disparate and fractured cloud environments. VMware has an answer for this. With VMware multi-cloud solutions, organizations have the choice, speed, and control to migrate and optimize
applications seamlessly without recoding, take the fastest path to modern infrastructure, and operate consistently across the data center, the edge, and any cloud. I urge to take a look at vmware.com/go/multicloud. You know my opinions on multi cloud by now, but there's a lot of stuff in here that works on any cloud. But don’t take it from me thats: vmware.com/go/multicloud and my thanks to them again for sponsoring my ridiculous nonsense.
Corey: You could build you go ahead and build your own coding and mapping notification system, but it takes time, and it sucks! Alternately, consider Courier, who is sponsoring this episode. They make it easy. You can call a single send API for all of your notifications and channels. You can control the complexity around routing, retries, and deliverability and simplify your notification sequences with automation rules. Visit courier.com today and get started for free. If you wind up talking to them, tell them I sent you and watch them wince—because everyone does when you bring up my name. Thats the glorious part of being me. Once again, you could build your own notification system but why on god’s flat earth would you do that?
Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. Roughly a year ago, I had a promoted guest episode featuring Ev Kontsevoy, the co-founder and CEO of Teleport.
A year has passed and what a year it’s been. Ev is back to tell us more about what they’ve been up to for the past year and, ideally, how things may have changed over in the security space. Ev, thank you for coming back to suffer the slings and arrows I will no doubt be hurling your way almost immediately.
Ev: Thanks for having me back, Corey.
Corey: So, it’s been a heck of a year. We were basically settling into the pandemic when last we recorded, and people’s security requirements when everyone is remote were dramatically changing. A year later, what’s changed? It seems like the frantic, grab a bucket and start bailing philosophy has largely been accepted with something that feels almost like a new normal, ish. What are you seeing?
Ev: Yes, we’re seeing exact same thing, that it’s really hard to tell what is normal. So, at the beginning of the pandemic, our company, Teleport was, so we were about 25 people. And then once we got the vaccines, and the government restrictions started to, kind of, disappear, people started to ask, “So, when are we going to go back to normal?” But the thing is, we’re 100 employees now, which means that three-quarters of the company, they joined us during the pandemic, so we have no normal to go back to. So, now we have to redefine—not redefined, we just basically need to get comfortable with this new, fully remote culture with fully remote identity that we have, and become comfortable with it. And that’s what we’re doing.
Corey: Beyond what, I guess, you’re seeing, as far as the culture goes, internally as well, it feels like there’s been a distinct shift in the past year or so, the entire security industry. I mean, I can sit here and talk about what I’ve seen, but again, I’m all over the place and I deal with a very select series of conversations. And I try not to confuse anecdotes with data. Anecdata is not the most reliable thing. You’re working in this space. That is the entire industry you’re in. How has the conversation in the industry around security shifted? What’s new? What trends are emerging?
Ev: So, there are several things actually happening. So, first of all, I wouldn’t call ourselves, like, we do all of security. So, we’re experts in access; like, how do you act this everything that you have in your cloud or in your data centers? And that space has been going through one transformation after another. It’s been basically under the same scaling stress as the rest of cloud computing industry.
And we can talk about historical changes that have been happening, and then we can talk a little bit about, kind of, latest and greatest. And in terms of what challenges companies have with secure access, maybe it helps if I just quickly describe what ‘access’ actually means.
Corey: Please, by all means. It’s one of those words that everyone knows, but if you ask three people to define it, you’ll get five definitions—
Ev: [laugh]. Exactly.
Corey: —and they don’t really align. So please, you’re the expert on this; I am here to listen because I guarantee you I am guilty of misusing the term at least once so far, today.
Ev: Can’t blame you. Can’t blame you. We are—I was same way until I got into this space. So, access basically means four things. So, if you want to have access done properly into your cloud resources, you need to think about four things.
First is connectivity. That’s basically a physical ability to deliver an encrypted packet from a client to destination, to a resource whatever that is, could be database, could be, like, SSH machine, or whatever it is you’re connecting to. So, connectivity is number one. So, then you need to authenticate. Authentication, that’s when the resource decides if you should have access or not, based on who you are, hopefully.
So, then authorization, that’s the third component. Authorization, the difference—like, sometimes people confuse the two—the difference between authentication and authorization is that authorization is when you already authenticated, but the resource decides what actions you are allowed to perform. The typical example is, like, is it read-only or read-write access? So, that’s authorization, deciding on which actions you’re allowed to perform. And the final component of having access properly is having audit or visibility which is, again, it could be real-time and historical.
So ideally, you need to have both. So, once you have those two solved, then you solved your access problem. And historically, if you look at how access has been done—so we had these giant machines, then we had microcomputers, then we had PCs, and they all have these things. So, you login into your Mac, and then if you try to delete certain file, you might get access denied. So, you see there is connectivity—in this case, it’s physical, a keyboard is physically connected to the [laugh] actual machine; so then you have authentication that you log in in the beginning; then authorization, if you can or cannot do certain things in your machine; and finally, your Mac keeps an audit log.
But then once the industry, we got the internet, we got all these clouds, so amount of these components that we’re now operating on, we have hundreds of thousands of servers, and load-balancers, and databases, and Kubernetes clusters, and dashboards, all of these things, all of them implement these four things: connectivity, authentication, authorization, audit.
Corey: Let me drive into that for a minute first, to make sure I’m clear on something. Connectivity makes sense. The network is the computer, et cetera. When you don’t have a network to something, it may as well not exist. I get that.
And the last one you mentioned, audit of a trail of who done it and who did what, when, that makes sense to me. But authentication and authorization are the two slippery ones in my mind that tend to converge a fair bit. Can you dive a little bit in delineate what the difference is between those two, please?
Ev: So authentication, if you try to authenticate into a database, database needs to check if you are on the list of people who should be allowed to access. That’s authentication, you need to prove that you are who you claim you are.
Corey: Do you have an account and credentials to get into that account?
Ev: Correct. And they’re good ways to do authentication and bad ways to do authentication. So, bad way to do authentication—and a lot of companies actually guilty of that—if you’re using shared credentials. Let’s say you have a user called ‘admin’ and that user has a password, and those are stored in some kind of stored—in, like 1Password, or something like Vault, some kind of encrypted Vault, and then when someone needs to access a database, they go and borrow this credentials and they go and do that. So, that is an awful way to do authentication.
Corey: Now, another way I’ve seen that’s terrible as been also, “Oh, if you’re connecting from this network, you must be allowed in,” which is just… yeee.
Ev: Oh, yeah. That’s a different sin. And that’s a perimeter security sin. But a much better way to do authentication is what is called identity-based authentication. Identity means that you always use your identity of who you are within the company.
So, you would go in through corporate SSO, something like Okta, or Active Directory, or even Google, or GitHub, and then based on that information, you’re given access. So, the resource in this case database, [unintelligible 00:07:39] say, “Oh, it’s Corey. And Corey is a member of this group, and also a member of that group.” And based on that it allows you to get in, but that’s where authentication ends. And now, if you want to do something, like let’s say you want to delete some data, now a database needs to check, ah, can you actually perform that action? That is the authorization process.
And to do that, usually, we use some mechanism like role-based access control. It will look into which group are you in. Oh, you are an admin, so admins have more privileges than regular people. So, then that’s the process of authorization.
And the importance of separating the two, and important to use identity because remember, audit is another important component of implementing access properly. So, if you’re sharing credentials, for example, you will see in your audit log, “Admin did this. Admin did that.” It’s exact same admin, but you don’t know who actually was behind that action. So, by sharing credentials, you’re also obscuring your own audit which is why it’s not really a good thing.
And going back to this industry trends is that because the amount of these resources, like databases and servers and so on, in the cloud has gotten so huge, so we now have this hardware pain, we just have too many things that need access. And all of these things, the software itself is getting more complicated, so now we have a software pain as well, that you have so many different layers in your stack that they need to access. That’s another dimension for introducing access pain. And also, we just have more developers, and the development teams are getting bigger and bigger, the software is eating the world, so there is a people-ware pain. So, on the one hand, you have these four problems you need to solve—connectivity, authentication, authorization, access—and on the other hand, you have more hardware, more software, more people, these pain points.
And so you need to consolidate, and that’s really what we do is that we allow you to have a single place where you can do connectivity, authentication, authorization, and audit, for everything that you have in the cloud. We basically believe that the future is going to be like metaverse, like in those books. So, all of these cloud resources are slowly converging into this one giant planetary-scale computer.
Corey: Suddenly, “I live on Twitter,” is no longer going to be quite as much of a metaphor as it is today.
Ev: [laugh]. No, no. Yeah, I think we’re getting better. If you look into what is actually happening on our computing devices that we buy, the answer is not the lot, so everything is running in data centers, the paradigm of thin client seems to be winning. Let’s just embrace that.
Corey: Yeah. You’re never going to be able to shove data centers worth compute into a phone. By the time you can get there, data centers will have gotten better. It’s the constant question of where do you want things to live? How do you want that to interact?
I talk periodically about multi-cloud, I talk about lock-in, everyone is concerned about vendor lock-in, but the thing that people tend to mostly ignore is that you’re already locked in throught a variety of different ways. And one way is both the networking side of it as well as the identity management piece because every cloud handles that differently and equating those same things between different providers that work different ways is monstrous. Is that the story of what you’re approaching from a Teleport perspective? Is that the primary use case, is that an ancillary use case, or are we thinking about this in too small a term?
Ev: So, you’re absolutely right, being locked in, in and—like, by itself is not a bad thing. It’s a trade-off. So, if you lack expertise in something and you outsourcing certain capability to a provider, then you’re developing that dependency, you may call it lock-in or not, but that needs to be a conscious decision. Like, well, you didn’t know how to do it, then someone else was doing it for you, so you should be okay with the lock-in. However, there is a danger, that, kind of, industry-wide danger about everyone relying on one single provider.
So, that is really what we all try to avoid. And with identity specifically, I feel like we’re in a really good spot that fairly early, I don’t see a single provider emerging as owning everyone’s identity. You know, some people use Okta; others totally happy tying everything to Google Apps. So, then you have people that rely on Amazon AWS native credentials, then plenty of smaller companies, they totally happy having all of their engineers authenticate through GitHub, so they use GitHub as a source of identity. And the fact that all of these providers are more or less compatible with each other—so we have protocols like OpenID Connect and SAML, so I’m not that concerned that identity itself is getting captured by a single player.
And Teleport is not even playing in that space; we don’t keep your identity. We integrate with everybody because, at the end of the day, we want to be the solution of choice for a company, regardless of which identity platform they’re using. And some of them using several, like all of the developers might be authenticating via GitHub, but everyone else goes through Google Apps, for example.
Corey: And the different product problem. Oh, my stars, I was at a relatively small startup going through an acquisition at one point in my career, and, “All right. Let’s list all of the SaaS vendors that we use.” And the answer was something on an average of five per employee by the time you did the numbers out, and—there were hundreds of them—and most of them because it started off small, and great, everyone has their own individual account, we set it up there. I mean, my identity management system here for what most of what I do is LastPass.
I have individual accounts there, two-factor auth enabled for anything that supports it, and that is it. Some vendors don’t support that: we have to use shared accounts, which is just terrifying. We make sure that we don’t use those for anything that’s important. But it comes down to, from our perspective, that everyone has their own ridiculous series of approaches, and even if we were to, “All right, it’s time to grow up and be a responsible business, and go for a single-sign-on approach.” Which is inevitable as companies scale, and there’s nothing wrong with that—but there’s still so many of these edge cases and corner case stories that don’t integrate.
So, it makes the problem smaller, but it’s still there rather persistently. And that doesn’t even get into the fact that for a lot of these tools, “Oh, you want SAML integration? Smells like enterprise to us.” And suddenly they wind up having an additional surcharge on top of that for accessing it via a federated source of identity, which means there are active incentives early on to not do that. So it’s—
Ev: It’s absolutely insane. Yeah, you’re right. You’re right. It’s almost like you get penalized for being small, like, in the early days. It’s not that easy if you have a small project you’re working on. Say it’s a company of three people and they’re just cranking in the garage, and it’s just so easy to default to using shared credentials and storing them in LastPass or 1Password. And then the interesting way—like, the longer you wait, the harder it is to go back to use a proper SSO for everything. Yeah.
Corey: I do want to call out that Teleport has a free and open-source community edition that supports GitHub SSO, and in order to support enterprise SSO, you have to go to your paid offering. I have no problem with this, to be clear, that you have to at least be our customer before we’ll integrate with your SSO solution makes perfect sense, but you don’t have a tiering system where, “Oh, you want to add that other SSO thing? And well, then it’s going to go from X dollars per employee to Y dollars.” Which is the path that I don’t like. I think it’s very reasonable to say that their features flat-out you don’t get as a free user. And even then you do offer SSO just not the one that some people will want to pick.
Ev: Correct. So, the open-source version of Teleport supports SSO that smaller companies use, versus our enterprise offering, we shaped it to be more appealing for companies at certain scale.
Corey: Yeah. And you’ve absolutely nailed it. There are a number of companies in the security space who enraged people about how they wind up doing their differentiation around things like SSO or, God forbid, two-factor auth, or once upon a time, SSL. This is not that problem. I just want to be explicitly clear on that, that is not what I’m talking about. But please, continue.
Ev: Look, we see it the same way. We sometimes say that we do not charge for security, like, top-level security you get, is available even in the open-source. And look, it’s a common problem for most startups who, when you have an open-source offering, where do you draw the line? And sometimes you can find answers in very unexpected places. For example, let’s look into security space.
One common reason that companies get compromised is, unfortunately, human factor. You could use the best tool in the world, but if you just by mistake, like, just put a comma in the wrong place and one of your config files just suddenly is out of shape, right, so—
Corey: People make mistakes and you can’t say, “Never make a mistake.” If you can get your entire company compromised by someone in your office clicking on the wrong link, the solution is not to teach people not to click on links; it’s to mitigate the damage and blast radius of someone clicking on a link that they shouldn’t. That is resilience that understand their human factors at play.
Ev: Yep, exactly. And here’s an enterprise feature that was basically given to us by customer requests. So, they would say we want to have FedRAMP compliance because we want to work with federal government, or maybe because we want to work with financial institutions who require us to have that level of compliance. And we tell them, “Yeah, sure. You can configure Teleport to be compliant. Look, here’s all the different things that you need to tweak in the config file.”
And the answer is, “Well, what if we make a mistake? It’s just too costly. Can we have Teleport just automatically works in that mode?” In other words, if you feed it the config file with an error, it will just refuse to work. So basically, you take your product, and you chop off things that are not compliant, which means that it’s impossible to feed an incorrect config file into it, and here you got an enterprise edition.
It’s a version that we call its FIPS mode. So, when it runs FIPS mode, it has different runtime inside, it basically doesn’t even have a crypto that
is not approved, which you can turn on by mistake. It will just not work.
Corey: By the time we’re talking about different levels of regulatory compliance, yeah, we are long past the point where I’m going to have any comments in the slightest is about differentiation of pricing tiers and the rest. Yeah, your free tier doesn’t support FedRAMP is one of those ludicrous things that—who would say that [laugh] actually be sincere [insane 00:18:28]?
Ev: [laugh].
Corey: That’s just mind-boggling to me.
Ev: Hold on a second. I don’t want anyone to be misinformed. You can be FedRAMP compliant with the free tier; you just need to configure it properly. Like the enterprise feature, in this case, we give you a thing that only works in this mode; it is impossible to misconfigure it.
Corey: It’s an attestation and it’s a control that you need—
Ev: Yep. Yep.
Corey: —in order to demonstrate compliance because half the joy of regulatory compliance is not doing the thing, it’s proving you do the thing. That is a joy, and those of you who’ve worked in regulated environments know exactly what I’m talking about. And those of you who have not, are happy but please—
Ev: Frankly, I think anyone can do it using some other open-source tools. You can even take, like, OpenSSH, sshd, and then you can probably build a different makefile for just the build pipeline that changes the linking, that it doesn’t even have the crypto that is not on the approved list. So, then if someone feeds a config file into it that has, like, a hashing function that is not approved, it will simply refuse to work. So, maybe you can even turn it into something that you could say here’s a hardened version of sshd, or whatever. So, same thing.
Corey: I see now you’re talking about the four aspects of this, the connectivity, the authentication, the authorization, and the audit components of access. How does that map to a software product, if that makes sense? Because it sounds like a series of principles, great, it’s good to understand and hold those in your head both, separately and distinct, but also combining to mean access both [technical 00:19:51] and the common parlance. How do you express that in Teleport?
Ev: So, Teleport doesn’t really add authorization, for example, to something that doesn’t have it natively. The problem that we have is just the overall increasing complexity of computing environments. So, when you’re deploying something into, let’s say, AWS East region, so what is it that you have there? You have some virtual machines, then you have something like Kubernetes on top, then you have Docker registry, so you have these containers running inside, then you have maybe MongoDB, then you might have some web UI to manage MongoDB and Grafana dashboard. So, all of that is software; we’re only consuming more and more of it so that our own code that we’re deploying, it’s icing on a
really, really tall cake.
And every layer in that layer cake is listening on a socket; it needs encryption; it has a login, so it has authentication; it has its own idea of role-based access control; it has its own config file. So, if you want to do cloud computing properly, so you got to have this expertise on your team, how to configure those four pillars of access for every layer in your stack. That is really the pain. And the Teleport value is that we’re letting you do it in one place. We’re saying, consolidate all of this four-axis pillars in one location.
That’s really what we do. It’s not like we invented a better way to authorize, or authenticate; no, we natively integrate with the cake, with all of these different layers. But consolidation, that is the key value of Teleport because we simply remove so much pain associated with configuring all of these things. Like, think of someone like—I’m trying not to disclose any names or customers, but let’s pick, uh, I don’t know, something like Tesla. So, Tesla has compute all over the world.
So, how can you implement authentication, authorization, audit log, and connectivity, too, for every vehicle that’s on the road? Because all of these things need software updates, they’re all components of a giant machine—
Corey: They’re all intermittent. You can’t say, “Oh, at this time of the day, we should absolutely make sure everything in the world is connected to the internet and ready to grab the update.” It doesn’t work that way; you’ve got to be… understand that connectivity is fickle.
Ev: So, most—and because computers growing generally, you could expect most companies in the future to be more like Tesla, so companies like that will probably want to look into Teleport technology.
Corey: This episode is sponsored in part by “you”—gabyte. Distributed technologies like Kubernetes are great, citation very much needed, because they make it easier to have resilient, scalable, systems. SQL databases haven’t kept pace though, certainly not like no SQL databases have like Route 53, the world’s greatest database. We’re still, other than that, using legacy monolithic databases that require ever growing instances of compute. Sometimes we’ll try and bolt them together to make them more resilient and scalable, but let’s be honest it never works out well. Consider Yugabyte DB, its a distributed SQL database that solves basically all of this. It is 100% open source, and there's not asterisk next to the “open” on that one. And its designed to be resilient and scalable out of the box so you don’t have to charge yourself to death. It's compatible with PostgreSQL, or “postgresqueal” as I insist on pronouncing it, so you can use it right away without having to learn a new language and refactor everything. And you can distribute it wherever your applications take you, from across availability zones to other regions or even other cloud providers should one of those happen to exist. Go to yugabyte.com, thats Y-U-G-A-B-Y-T-E dot com and try their free beta of Yugabyte Cloud, where they host and manage it for you. Or see what the open source project looks like—its effortless distributed SQL for global apps. My thanks to Yu—gabyte for sponsoring this episode.
Corey: If we take a look at the four tenets that you’ve identified—connectivity, authentication, authorization, and audit—it makes perfect sense. It is something that goes back to the days when computers were basically glorified pocket calculators as opposed to my pocket calculator now being basically a supercomputer. Does that change as you hit cloud-scale where we have companies that are doing what seem to be relatively pedestrian things, but also having 100,000 EC2 instances hanging out in AWS? Does this add additional levels of complexity on top of those four things?
Ev: Yes. So, there is one that I should have mentioned earlier. So, in addition to software, hardware, and people-ware—so those are three things that are exploding, more compute, more software, more engineers needing access—there is one more dimension that is kind of unique, now, at the scale that we’re in today, and that’s time. So, let’s just say that you are a member of really privileged group like you’re a DBA, or maybe you are a chief security officer, so you should have access to a certain privileged database. But do you really use that access 24/7, all the time? No, but you have it.
So, your laptop has an ability, if you type certain things into it, to actually receive credentials, like, certificates to go and talk to this database all the time. It’s an anti-pattern that is now getting noticed. So, the new approach to access is to make a tie to an intent. So, by default, no one in an organization has access to anything. So, if you want to access a database, or a server, or Kubernetes cluster, you need to issue what’s called ‘access request.’
It’s similar to pull request if you’re trying to commit code into Git. So, you send an access request—using Teleport for example; you could probably do it some other way—and it will go into something like Slack or PagerDuty, so your team members will see that, “Oh, Corey is trying to access that database, and he listed a ticket number, like, some issue he is trying to troubleshoot with that particular database instance. Yeah, we’ll approve access for 30 minutes.” So, then you go and do that, and the access is revoked automatically after 30 minutes. So, that is this new trend that’s happening in our space, and it makes you feel nice, too, it means that if someone hacks into your laptop at this very second, right after you finished authenticating and authorization, you’re still okay because there is no access; access will be created for you if you request it based on the intent, so it dramatically reduces the attack surface, using time as additional dimension.
Corey: The minimum viable permission to do a thing. In principle, least-access is important in these areas. It’s like, “Oh, yeah, my user account, you mean root?” “Yeah, I guess that works in a developer environment,” looks like a Docker container that will be done as soon as you’re finished, but for most use cases—and probably even that one—that’s not the direction to go in. Having things scoped down and—
Ev: Exactly.
Corey: —not just by what the permission is, but by time.
Ev: Exactly.
Corey: Yeah.
Ev: This system basically allows you to move away from root-type accounts completely, for everything. So, which means that there is no root to attack anymore.
Corey: What really strikes me is how, I guess, different aspects of technology that this winds up getting to. And to illustrate that in the form of question, let me go back to my own history because, you know, let’s make it about me here. I’ve mentioned it before on the show, but I started off my technical career as someone who specialized in large-scale email systems. That was a niche I found really interesting, and I got into it. So did you.
I worked on running email servers, and you were the CEO and co-founder of Mailgun, which later you sold the Rackspace. You’re a slightly bigger scale than I am, but it was clear to me that even then, in the 2006 era when I was doing this, that there was not going to be the same need going forward for an email admin at every company; the cloudification of email had begun, and I realized I could either dig my heels in and fight the tide, or I could find other things to specialize in. And I’ve told that part of the story, but what I haven’t told is that it was challenging at first as I tried to do that because all the jobs I talked to looked at my resume and said, “Ah, you’re the email admin. Great. We don’t need one of those.”
It was a matter of almost being pigeonholed or boxed into the idea of being the email person. I would argue that Teleport is not synonymous with email in any meaningful sense as far as how it is perceived in the industry; you are very clearly no longer the email guy. Does the idea being boxed in, I guess—
Ev: [laugh].
Corey: —[unintelligible 00:27:05] resonate at all with you? And if so, how did you get past it?
Ev: Absolutely. The interesting thing is, before starting the Mailgun, I was not an email person. I would just say that I was just general-purpose technologist, and I always enjoyed building infrastructure frameworks. Basically, I always enjoyed building tools for other engineers. But then gotten into this email space, and even though Mailgun was a software product, which actually had surprisingly huge, kind of, scalability requirements early on because email is much heavier than HTTP traffic; people just send a lot of data via emails.
So, we were solving interesting technical challenges, but when I would meet other engineers, I would experience the exact same thing you did. They would put me into this box of, “That’s an email guy. He knows email technology, but seemingly doesn’t know much about scaling web apps.” Which was totally not true. And it bothered me a little bit.
Frankly, it was one of the reasons we decided to get acquired by Rackspace because they effectively said, “Why don’t you come join us and we’ll continue to operate as independent company, but you can join our cloud team and help us reinvent cloud computing.” It was really appealing. So, I actually moved to Texas after acquisition; I worked on the Rackspace cloud team for a while. So, that’s how my transition from this being in the email box happened. So, I went from an email expert to just generally cloud computing expert. And cloud computing expert sounds awesome, and it allows me to work—
Corey: I promise, it’s not awesome—
Ev: [laugh].
Corey: —for people listening to this. Also, it’s one of those, are you a cloud expert? Everyone says no to that because who in the world would claim that? It’s so broad in so many different expressions of it. Because you know the follow-up question to anyone who says, “Yeah,” is going to be some esoteric thing about a system you’ve never heard of before because there’s so many ridiculous services across totally different providers, of course, it’s probably a thing. Maybe it’s actually a Pokemon, we don’t know. But it’s hard to consider yourself an expert in this. It’s like, “Well, I have some damage from [laugh] getting smacked around by clouds and, yeah, we’ll call that expertise; why not?”
Ev: Exactly. And also how frequently people mispronounce, like, cloud with clown. And it’s like, “Oh, I’m clown computing expert.” [laugh].
Corey: People mostly call me a loud computing expert. But that’s a separate problem.
Ev: But the point is that if you work on a product that’s called cloud, so you definitely get to claim expertise of that. And the interesting thing that Mailgun being, effectively, an infrastructure-level product—so it’s part of the platform—every company builds their own cloud platform and runs it, and so Teleport is part of that. So, that allowed us to get out of the box. So, if you working on, right now we’re in the access space, so we’re working closely with Kubernetes community, with Linux kernel community, with databases, so by extension, we have expertise in all of these different areas, and it actually feels much nicer. So, if you are computing security access company, people tend to look at you, it’s like, “Yeah, you know, a little bit of everything.” So, that feels pretty nice.
Corey: It’s of those cross-functional things—
Ev: Yeah, yeah.
Corey: —whereas on some level, you just assume, well, email isn’t either, but let’s face it: email is the default API that everything, there’s very little that you cannot configure to send email. The hard part is how to get them to stop emailing you. But it started off as far—from my world at least—the idea that all roads lead to email. In fact, we want to talk security, a long time ago the internet collectively decided one day that our email inbox was the entire cornerstone of our online identity. Give me access to your email, I, for all intents and purposes, can become you on the internet without some serious controls around this.
So, those conversations, I feel like they were heading in that direction by the time I left email world, but it’s very clear to me that what you’re doing now at Teleport is a much clearer ability to cross boundaries into other areas where you have to touch an awful lot of different things because security touches everything, and I still maintain it has to be baked-in and an intentional thing, rather than, “Oh yeah, we’re going to bolt security on after the fact.” It’s, yeah, you hear about companies that do that, usually in headlines about data breaches, or worse. It’s a hard problem.
Ev: Actually, it’s an interesting dilemma you’re talking about. Is security built-in into everything or is it an add-on? And logically—talk to anyone, and most people say, “Yeah, it needs to be a core component of whatever it is you’re building; making security as an add-on is not possible.” But then reality hits in, and the reality is that we’re running on—we’re standing on the shoulder of giants.
There is so much legacy technologies that we built this cloud monster on top of… no, nothing was built in, so we actually need to be very crafty at adding security on top of what we already have, if we want to take advantage of all this pre-existing things that we’ve built for decades. So, that’s really what’s happening, I think, with security and access. So, if you ask me if Teleport is a bolt-on security, I say, “Yes, we are, but it works really well.” And it’s extremely pragmatic and reasonable, and it gives you security compliance, but most of all, very, very good user experience out of the box.
Corey: It’s amazing to me how few security products focus on user experience out of the box, but they have to. You cannot launch or maintain a security product successfully—to my mind—without making it non-adversarial to the user. The [days of security is no 00:32:26] are gone.
Ev: Because of that human element insecurity. If you make something complicated, if you make something that’s hard to reason about, then it will never be secure.
Corey: Yeah.
Ev: Don’t copy-paste IP table rules without understanding what they do. [laugh].
Corey: Yeah, I think we all have been around long enough in data center universes remember those middle of the night drives to the data center for exactly that sort of thing. Yeah, it’s one of those hindsight things of, set a cron job to reset the IP table rules for, you know, ten minutes from now in case you get this hilariously wrong. It’s the sort of thing that you learn right after you really could have used that knowledge. Same story. But those are the easy, safe examples of I screwed up on a security thing. The worst ones can be company-ending.
Ev: Exactly, yeah. So, in this sense, when it comes to security, and access specifically, so this old Python rule that there is only one way to do something, it’s the most important thing you can do. So, when it comes to security and access, we basically—it’s one of the things that Teleport is designed around, that for all protocols, for all different resources, from SSH to Kubernetes to web apps to databases; we never support passwords. It’s not even in the codebase. No, you cannot configure Teleport to use passwords.
We never support things like public keys, for example, because it’s just another form of a password. It’s just extremely long password. So, we have this approach that certificates, it’s the best method because it supports both authentication and authorization, and then you have to do it for everything, just one way of doing everything. And then you apply this to connectivity: so there is a single proxy that speaks all protocols and everyone goes to that proxy. Then you apply the same principle to audit: there is one audit where everything goes into.
So, that’s how this consolidation, that’s where the simplicity comes down to. So, one way of doing something; one way of configuring everything. So, that’s where you get both ease of use and security at the same time.
Corey: One last question that I want to ask you before we wind up calling this an episode is that I’ve been using Teleport as a reference for a while when I talk to companies, generally in the security space, as an example of what you can do to tell a story about a product that isn’t built on fear, uncertainty, and doubt. And for those who are listening who don’t know what I’m referring specifically, I’m talking about pick any random security company and pull up their website and see what it is that they talk about and how they talk about themselves. Very often, you’ll see stories where, “Data breaches will cost you extraordinary piles of money,” or they’ll play into the shame of what will happen to your career if you’re named in the New York Times for being the CSO when the data gets breached, and whatnot. But everything that I’ve seen from Teleport to date has instead not even gone slightly in that direction; it talks again and again, in what I see on your site, about how quickly it is to access things, access that doesn’t get in the way, easily implement security and compliance, visibility into access and behavior. It’s all about user experience and smoothing the way and not explaining to people what the dire problems that they’re going to face are if they don’t care about security in general and buy your product specifically. It is such a refreshing way of viewing storytelling around a security product. How did you get there? And how do I make other people do it, too?
Ev: I think it just happened organically. Teleport originally—the interesting story of Teleport, it was not built to be sold. Teleport was built as a side project that we started for another system that we were working on at the time. So, there was a autonomous Kubernetes platform called Grá—it doesn’t really matter in this context, but we had this problem that we had a lot of remote sites with a lot of infrastructure on them, with extremely strict security and compliance requirements, and we needed to access those sites or build tools to access those sites. So, Teleport was built like, okay, it’s way better than just stitching a bunch of open-source components together because it’s faster and easier to use, so we’re optimizing for that.
And as a side effect of that simplification, consolidation, and better user experience is a security compliance. And then the interesting thing that happened is that people who we’re trying to sell the big platform to, they started to notice about, “Oh, this access thing you have is actually pretty awesome. Can we just use that separately?” And that’s how it turned into a product. So, we built an amazing secure access solution almost by accident because there was only one customer in mind, and that was us, in the early days. So yeah, that’s how you do it, [laugh] basically. But it’s surprisingly similar to Slack, right? Why is Slack awesome? Because the team behind it was a gaming company in the beginning.
Corey: They were trying to build a game. Yeah.
Ev: Yeah, they built for themselves. They—[laugh] I guess that’s the trick: make yourself happy.
Corey: I think the team founded Flickr before that, and they were trying to build a game. And like, the joke I heard is, like, “All right, the year is 2040. Stuart and his team have now raised $8 billion trying to build a game, and yet again it fails upward into another productivity tool company, or something else entirely that”—but it’s a recurring pattern. Someday they’ll get their game made; I have faith in them. But yeah, building a tool that scratches your own itch is either a great path or a terrible mistake, depending entirely upon whether you first check and see if there’s an existing solution that solves the problem for you. The failure mode of this is, “Ah, we’re going to build our own database engine,” in almost every case.
Ev: Yeah. So just, kind of like, interesting story about the two, people will [unintelligible 00:38:07] surprised that Teleport is a single binary. It’s basically a drop-in replacement that you put on a box, and it runs instead of sshd. But it wasn’t initially this way. Initially, it was [unintelligible 00:38:16], like, few files in different parts of a file system. But because internally, I really wanted to run it on a bunch of Raspberry Pi’s at home, and it would have been a lot easier if it was just a single file because then I just could quickly update them all. So, it just took a little bit of effort to compress it down to a single binary that can run in different modes depending on the key. And now look at that; it’s a major benefit that a lot of people who deploy Teleport on hundreds of thousands of pieces of infrastructure, they definitely taking advantage of the fact that it’s that simple.
Corey: Simplicity is the only thing that scales. As soon as it gets complex, it’s more things to break. Ev, thank you so much for taking the time to sit with me, yet again, to talk about Teleport and how you’re approaching things. If people want to learn more about you, about the company, about the product in all likelihood, where can they go?
Ev: The easiest place to go would be goteleport.com where you can find everything, but we’re also on GitHub. If you search for Teleport in GitHub, you’ll find this there. So, join our Slack channel, join our community mailing list and most importantly, download Teleport, put it on your Raspberry Pi, play with it and see how awesome it is to have the best industry, best security practice, that don’t get in the way.
Corey: I love the tagline. Thank you so much, once again. Ev Kontsevoy, co-founder and CEO of Teleport. I’m Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice along with a comment that goes into a deranged rant about how I’m completely wrong, and the only way to sell security products—specifically yours—is by threatening me with the New York Times data breach story.
Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.
Announcer: This has been a HumblePod production. Stay humble.