Subscribe
Share
Share
Embed
Screaming in the Cloud with Corey Quinn features conversations with domain experts in the world of Cloud Computing. Topics discussed include AWS, GCP, Azure, Oracle Cloud, and the "why" behind how businesses are coming to think about the Cloud.
Transcript
Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. My guest today is Jeremy Snyder, who’s the founder at Firetail. Jeremy, thank you for joining me today. I appreciate you taking the time from your day to suffer my slings and arrows.
Jeremy: My pleasure, Corey. I’m really happy to be here.
Corey: So, we’ll get to a point where we talk about what you’re up to these days, but first, I want to dive into the jobs of yesteryear because over a decade ago, you did a stint at AWS doing sales. And not to besmirch your hard work, but it feels like at the time, that must have been a very easy job. Because back then it really felt across the board like the sales motion was basically responding to, “Well, why should we do business with you?” And the response is, “Oh, you misunderstand. You have 87 different accounts scattered throughout your organization. I’m just here to give you visibility, governance, and possibly some discounting over that.” It feels like times have changed in a lot of ways since then. Is that accurate?
Jeremy: Well, yeah, but I will correct a couple of things in there. In my days—
Corey: Oh, please.
Jeremy: —almost nobody had more than one account. I was in the one account, no VPCs, you know, you only separate your workloads by tagging days of AWS. So, our job was a lot, actually, harder at the time because people couldn’t wrap their heads around the lack of subnetting, the lack of workload segregation. All of that was really, like, brand new to people, and so you were trying to tell them like, “Hey, you’re going to be launching something on an EC2 instance that’s in the same subnet as everybody else’s EC2 instance.” And people were really worried about lateral traffic and sniffing and what could their neighbors or other customers on AWS see. And by the way, I mean, this was the customers who even believed it was real. You know, a lot of the conversations we went into with people was, “Oh, so Amazon bought too many servers and you’re trying to sell us excess capacity.”
Corey: That legend refuses to die.
Jeremy: And, you know, it is a legend. That is not at all the genesis of AWS. And you know, the genesis is pretty well publicized at this point; you can go just google, “how did AWS started?” You can find accurate stuff around that.
Corey: I did it a few years ago with multiple Amazon execs and published it, and they said definitively that that story was not true. And you can say a lot about AWS folks, and I assure you, I do, but I also do not catch them lying to my face, ever. And as soon as that changes, well, now we’re going to have a different series of [laugh] conversations that are a lot more pointed. But they’ve earned some trust there.
Jeremy: Yeah, I would agree. And I mean, look, I saw it internally, the way that Amazon built stuff was at such a breakneck pace, that challenge that they had that was, you know, the published version of events for why AWS got created, developers needed a place to test code. And that was something that they could not get until they got EC2, or could not get in a reasonably enough timeframe for it to be, you know, real-time valid or relevant for what was going on with the company. So, you know, that really is the genesis of things, and you know, the early services, SQS, S3, EC2, they all really came out of that journey. But yeah, in our days at AWS, there was a lot of ease, in the sense that lots of customers had pent-up frustrations with their data center providers or their colo providers and lots of customers would experience bursts and they would have capacity constraints and they would need a lot of the features that AWS offered, but we had to overcome a lot of technical misunderstandings and trust issues and, you know, oh, hey, Amazon just wants to sniff our data and they want to see what we’re up to, and explain to them how encryption works and why they have their own keys and all these things. You know, we had to go through a lot of that. So, it wasn’t super easy, but there was some element of it where, you know, just demand actually did make some aspects easy.
Corey: What have you seen change since, well I guess ten years ago and change now? And let’s be clear, you don’t work in AWS sales, but you also are not oblivious to what the market is doing.
Jeremy: For sure. For sure. I left AWS in 2011 and I’ve stayed in the cloud ecosystem pretty much ever since. I did spend some time working for a system integrator where all we did was migrate customers to AWS. And then I spent about five, six years working on cloud security primarily focused on AWS, a lot of GCP, a little bit of Azure.
So yeah, I mean, I certainly stay up to date with what’s going on in the state of cloud. I mean, look, Cloud has evolved from this kind of, you know, developer-centric, very easy-to-launch type of platform into a fully-fledged enterprise IT platform and all of the management structures and all of the kind of bells and whistles that you would want that you probably wanted from your old VMware networks but never really got, they’re all there now. It is a very different ballgame in terms of what the platform actually enables you to do, but fundamentally, a lot of the core building block constructs and the primitives are still kind of driving the heart of it. It’s just a lot of nicer packaging.
What I think is really interesting is actually how customers’ usage of cloud platforms has changed over time. And I always think of it and kind of like the, going back to my days, what did I see from my customers? And it was kind of like the month zero, “I just don’t believe you.” Like, “This thing can’t be real, I don’t trust it, et cetera.” Month one is, I’m going to assign some developer to work on some very low-priority, low-risk workload. In my days, that was SharePoint, by the way. Like, nine times out of ten, the first workload that customers stood up was a SharePoint instance that they had to share across multiple locations.
Corey: That thing falls over all the time anyway. May as well put it in the cloud where it can do so without taking too much else down with it. Was that the thinking or?
Jeremy: Well, and the other thing about it at the time, Corey, was that, like, so many customers worked in this, like, remote-first world, right? And so, SharePoint was inevitably hosted at somebody’s office. And so, the workers at that office were so privileged over the workers everywhere else. The performance gap between consuming SharePoint in one location versus another was like, night and day. So, you know, employees in headquarters were like, “Yeah, SharePoint’s great.” Employees in branch offices were like, “This thing is terrible,” you know? “It’s so slow. I hate it, I hate it, I hate it.”
And so, Cloud actually became, like, this neutral location to move SharePoint to that kind of had an equal performance for every office. And so, that was, I think, one of the reasons and it was also, you know, it had capacity problems, and customers were right at that point, uploading tons of static documents to it, like Word documents, Office attachments, et cetera, and so they were starting to have some of these, like, real disk sprawl problems with SharePoint. So, that was kind of the month one problem. And only after they get through kind of month two, three, and four, and they go through, “I don’t understand my bill,” and, “Help me understand security implications,” then they think about, like, “Hey, should we go back and look at how we’re running that SharePoint stuff and maybe do it more efficiently and, like, move those static Office documents onto S3?” And so on, and so on.
And that’s kind of one of the big things that I’ve changed that I would say is very different from, like, 2011 to now, is there’s enough sophistication around understanding that, like, you don’t just translate what you’re doing in your office or in your data center to what you’re doing on cloud. Or if you do, you’re not getting the most out of your investment.
Corey: I’m curious to get your take on how you have seen cloud adoption patterns differ, specifically tied to geo. I mean, I tend to see it from a world where there’s a bifurcation of between born-in-the-cloud SaaS-type companies where one workload is 80% of their bill or whatnot, and of the big enterprises where the largest single component is 3%. So, it’s a very different slice there. But I’m curious what you would see from a sales perspective, looking across a lot of different geographic boundaries because we’re all, on some level, biased based upon where we tend to spend our time doing business. I’m in San Francisco, which is its very own strange universe that has a certain perspective about itself that is occasionally accurate, but not usually. But it’s a big world out there.
Jeremy: It is. One thing that I would say it’s interesting. I spent my AWS days based in Singapore, living in Singapore at the time, and I was working with customers across Southeast Asia. And to your point, Corey, one of the most interesting things was this little bit of a leapfrog effect. Data centers in Asia-Pac, especially in places like the Philippines, were just terrible.
You know, the Philippines had, like, the second highest electricity rates in Asia at the time, only behind Japan, even though the GDP per capita gap between those two countries is really large. And yet you’re paying, like, these super-high electricity rates. Secondarily, data centers in the Philippines were prone to flooding. And so, a lot of companies in the Philippines never went the data center route. You know, they just hosted servers in their offices, you know, they had a bunch of desktop machines in a cubicle, that kind of situation because, like, data centers themselves were cost prohibitive.
So, you saw this effect a little bit like cell phones in a lot of the developing world. Landline infrastructure was too expensive or never got done for whatever reason, and people went straight to cell phones. So actually, what I saw in a lot of emerging markets in Asia was, screw the data center; we’re going to go straight to cloud. So, I saw a lot of Asia-Pac get a little bit ahead of places like Europe where you had, for instance, a lot of long-term data center contracts and you had customers really locked in. And we saw this over the next, let’s say between, like, say, 2014 and 2018 when I was working with a systems integrator, and then started working on cloud security.
We saw that US customers and Asia-Pac customers didn’t have these obligations; European customers, a lot of them were still working off their lease, and still, you know, I’m locked into let’s just say Equinix Frankfurt for another five years before I can think about cloud migration. So, that’s definitely one aspect that I observed. Second thing I think is, like, the earlier you started, the earlier you reached the point where you realize that actually there is value in a lot of managed services and there actually is value in getting away from the kind of server mindset around EC2.
Corey: It feels like there’s a lot of, I want to call it legacy thinking, in some ways, except that’s unfair because legacy remains a condescending engineering term for something that makes money. The problem that you have is that you get bound by choices you didn’t necessarily realize you were making, and then something becomes revenue-bearing. And now there’s a different way to do it, or you learn more about the platform, or the platform itself evolves, and, “Oh, I’m going to rewrite everything to take advantage of this,” isn’t happening. So, it winds up feeling like, yeah, we’re treating the cloud like a data center. And sometimes that’s right; sometimes that’s a problem, but ultimately, it still becomes a significant challenge. I mean, there’s no way around it. And I don’t know what the right answer is, I don’t know what the fix is going to be, but it always feels like I’m doing something wrong somewhere.
Jeremy: I think a lot of customers go through that same set of feelings and they realize that they have the active runway problem, where you know, how do you do maintenance on an active runway? You kind of can’t because you’ve got flights going in and out. And I think you’re seeing this in your part of the world at SFO with a lot of the work that got done in, like, 2018, 2019 where they kind of had to close down a runway and had, like, near misses because they consolidated all flights onto the one active runway, right? It is a challenge. And I actually think that some of the evolution that I’ve seen our customers go through over the last, like, two, three years, is starting to get away from that challenge.
So, to your point, when you have revenue-bearing workloads that you can’t really modify and things are pretty tightly coupled, it is very hard to make change. But when you start to have it where things are broken down into more microservices, it makes it a lot easier to cycle out Service A for Service B, or let’s say more accurately, Service A1 with Service A2 where you can kind of just, like, plug and play different APIs, and maybe, you know, repoint services at the new stuff as they come online. But getting to that point is definitely a painful process. It does require architectural changes and often those architectural changes aren’t at the infrastructure level; they’re actually inside the application or they’re between things like applications and third-party dependencies where the customers may not have full control over the dependencies, and that does become a real challenge for people to break down and start to attack. You’ve heard of the Strangler Methodology?
Corey: Oh, yes. Both in terms of the Boston Strangler, as well—
Jeremy: [laugh]. Right.
Corey: As the Strangler design pattern.
Jeremy: Yeah, yeah. But I think, like, getting to that is challenging until, like, once you understand that you want to do that, it makes a lot of sense. But getting to the starting point for that journey can be really challenging for a lot of customers because it involves stakeholders that are often not involved on infrastructure conversations, and organizational dysfunction can really creep in there, where you have teams that don’t necessarily play nice together, not for any particular reason, but just because historically they haven’t had to. So, that’s something that I’ve seen and definitely takes a little bit of cultural work to overcome.
Corey: When you take a look across the board of cloud adoption, it’s interesting to have seen the patterns that wound up unfolding. Your career path, though, seem to have gotten away from the selling cloud and into some strange directions leading up to what you’re doing now, where you founded Firetail. What do you folks do?
Jeremy: We do API security. And it really is kind of the culmination of, like, the last several years and what we saw. I mean, to your point, we saw customers going through kind of Phase One, Two, Three of cloud adoption. Phase One, the, you know, for lack of a better phrase, lift-and-shift and Phase Two, the kind of first step on the path towards quote-unquote, “Enlightenment,” where they start to see that, like, actually, we can get better operational efficiency if we, you know, move our databases off of EC2 and on to RDS and we move our static content onto S3.
And then Phase Three, where they realize actually EC2 kind of sucks, and it’s a lot of management overhead, it’s a lot of attack surface, I hate having to bake AMIs. What I really want to do is just drop some code on a platform and run my application. And that might be serverless. That might be containerized, et cetera. But one path or the other, where we pretty much always see customers ending up is with an API sitting on a network.
And that API is doing two things. It front-ends a data set and at front-ends a set of functionality, and most cases. And so, what that really means is that the thing that sits on the network that does represent the attack surface, both in terms of accessing data or in terms of let’s say, like, abusing an application is an API. And that’s what led us to where I am today, what led me and my co-founder Riley to, you know, start the company and try to make it easier for customers to build more secure APIs. So yeah, that’s kind of the change that I’ve observed over the last few years that really, as you said, lead to what I’m doing now.
Corey: There is a lot of, I guess, challenge in the entire space when we bound that to—even API security, though as soon as you going down the security path it starts seeming like there’s a massive problem, just in terms of proliferation of companies that each do different things, that each focus on different parts of the story. It feels like everything winds up spitting out huge amounts of security-focused, or at least security-adjacent telemetry. Everything has findings on top of that, and at least in the AWS universe, “Oh, we have a service that spits out a lot of that stuff. We’re going to launch another service on top of it that, of course, cost more money that then winds up organizing it for you. And then another service on top of that that does the same thing yet again.” And it feels like we’re building a tower of these things that are just… shouldn’t just be a feature in the original underlying thing that turns down the noise? “Well, yes, but then we couldn’t sell you three more things around it.”
Jeremy: Yeah, I mean—
Corey: Agree? Disagree?
Jeremy: I don’t entirely disagree. I think there is a lot of validity on what you just said there. I mean, if you look at like the proliferation of even the security services, and you see GuardDuty and Config and Security Hub, or things like log analysis with Athena or log analysis with an ELK stack, or OpenSearch, et cetera, I mean, you see all these proliferation of services around that. I do think the thing to bear in mind is that for most customers, like, security is not a one size fits all. Security is fundamentally kind of a risk management exercise, right? If it wasn’t a risk management exercise, then all security would really be about is, like, keeping your data off of networks and making sure that, like, none of your data could ever leave.
But that’s not how companies work. They do interact with the outside world and so then you kind of always have this decision and this trade-off to make about how much data you expose. And so, when you have that decision, then it leads you down a path of determining what data is important to your organization and what would be most critical if it were breached. And so, the point of all of that is honestly that, like, security is not the same for you as it is for me, right? And so, to that end, you might be all about Security Hub, and Config instead of basic checks across all your accounts and all your active regions, and I might be much more about, let’s say I’m quote-unquote, “Digital-native, cloud-native,” blah, blah, blah, I really care about detection and response on top of events.
And so, I only care about log aggregation and, let’s say, GuardDuty or Athena analysis on top of that because I feel like I’ve got all of my security configurations in Infrastructure as Code. So, there’s not a right and wrong answer and I do think that’s part of why there are a gazillion security services out there.
Corey: On some level, I’ve been of the opinion for a while now that the cloud providers themselves should not necessarily be selling security services directly because, on some level, that becomes an inherent conflict of interest. Why make the underlying platform more secure or easier to use from a security standpoint when you can now turn that into a revenue source? I used to make comments that Microsoft Defender was a classic example of getting this right because they didn’t charge for it and a bunch of antivirus companies screamed and whined about it. And then of course, Microsoft’s like, “Oh, Corey saying nice things about us. We can’t have that.” And they started charging for it. So okay, that more or less completely subverts my entire point. But it still feels squicky.
Jeremy: I mean, I kind of doubt that’s why they started charging for it. But—
Corey: Oh, I refuse to accept that I’m not that influential. There we are.
Jeremy: [laugh]. Fair enough.
Corey: Yeah, I just can’t get away from the idea that it feels squicky when the company providing the infrastructure now makes doing the secure thing on top of it into an investment decision.
Jeremy: Yeah.
Corey: “Do you want the crappy, insecure version of what we build or do you want the top-of-the-line secure version?” That shouldn’t be a choice people have to make. Because people don’t care about security until right after they really should have cared about security.
Jeremy: Yeah. Look, and I think the changes to S3 configuration, for instance, kind of bear out your point. Like, it shouldn’t be the case that you have to go through a lot of extra steps to not make your S3 data public, it should always be the case that, like, you have to go through a lot of steps if you want to expose your data. And then you have explicitly made a set of choices on your own to make some data public, right? So, I kind of agree with the underlying logic. I think the counterargument, if there is one to be made, is that it’s not up to them to define what is and is not right for your organization.
Because again, going back to my example, what is secure for you may not be secure for me because we might have very different modes of operation, we might have very different modes of building our infrastructure, deploying our infrastructure, et cetera. And I think every cloud provider would tell you, “Hey, we’re just here to enable customers.” Now, do I think that they could be doing more? Do I think that they could have more secure defaults? You know, in general, yes, of course, they could. And really, like, the fundamentals of what I worry about are people building insecure applications, not so much people deploying infrastructure with bad configurations.
Corey: It’s funny, we talk about this now. Earlier today, I was lamenting some of the detritus from some of my earlier builds, where I’ve been running some of these things in my old legacy single account for a while now. And the build service is dramatically overscoped, just because trying to get the security permissions right, was an exercise in frustration at the time. It was, “Nope, that’s not it. Nope, blocked again.”
So, I finally said to hell with it, overscope it massively, and then with a, “Todo: fix this later,” which of course, never happened. And if there’s ever a breach on something like that, I know that I’ll have AWS wagging its finger at me and talking about the shared responsibility model, but it’s really kind of a disaster plan of their own making because there’s not a great way to say easily and explicitly—or honestly, by default the way Google Cloud does—of okay, by default, everything in this project can talk to everything in this project, but the outside world can’t talk to any of it, which I think is where a lot of people start off. And the security purists love to say, “That’s terrible. That won’t work at a bank.” You’re right, it won’t, but a bank has a dedicated security apparatus, internally. They can address those things, whereas your individual student learner does not. And that’s how you wind up with open S3 bucket monstrosities left and right.
Jeremy: I think a lot of security fundamentalists would say that what you just described about that Google project structure, defeats zero trust, and you know, that on its own is actually a bad thing. I might counterargue and say that, like, hey, you can have a GCP project as a zero trust, like, first principle, you know? That can be the building block of zero trust for your organization and then it’s up to you to explicitly create these trust relationships to other projects, and so on. But the thing that I think in what you said that really kind of does resonate with me in particular as an area that AWS—and really this case, just AWS—should have done better or should do better, is IAM permissions. Because every developer in the world that I know has had that exact experience that you described, which is, they get to a point where they’re like, “Okay, this thing isn’t working. It’s probably something with IAM.”
And then they try one thing, two things, and usually on the third or fourth try, they end up with a star permission, and maybe a comment in that IAM policy or maybe a Jira ticket that, you know, gets filed into backlog of, “Review those permissions at some point in the future,” which pretty much never happens. So, IAM in particular, I think, is one where, like, Amazon should do better, or should at least make it, like, easy for us to kind of graphically build an IAM policy that is scoped to least permissions required, et cetera. That one, I’ll a hundred percent agree with your comments and your statement.
Corey: As you take a look across the largest, I guess, environments you see, and as well as some of the folks who are just getting started in this space, it feels like, on some level, it’s two different universes. Do you see points of commonality? Do you see that there is an opportunity to get the individual learner who’s just starting on their cloud journey to do things that make sense without breaking the bank that they then can basically have instilled in them as they start scaling up as they enter corporate environments where security budgets are different orders of magnitude? Because it seems to me that my options for everything that I’ve looked at start at tens of thousands of dollars a year, or are a bunch of crappy things I find on GitHub somewhere. And it feels like there should be something between those two.
Jeremy: In terms of training, or in terms of, like, tooling to build—
Corey: In terms of security software across the board, which I know—
Jeremy: Yeah.
Corey: —is sort of a vague term. Like, I first discovered this when trying to find something to make sense of CloudTrail logs. It was a bunch of sketchy things off GitHub or a bunch of very expensive products. Same thing with VPC flow logs, same thing with trying to parse other security alerting and aggregate things in a sensible way. Like, very often it’s, oh, there’s a few very damning log lines surrounded by a million lines of nonsense that no one’s going to look through. It’s the needle in a haystack problem.
Jeremy: Yeah, well, I’m really sorry if you spent much time trying to analyze VPC flow logs because that is just an exercise in futility. First of all, the level of information that’s in them is pretty useless, and the SLA on actually, like, log delivery, A, whether it’ll actually happen, and B, whether it will happen in a timely fashion is just pretty much non-existent. So—
Corey: Oh, from a security perspective I agree wholeheartedly, but remember, I’m coming from a billing perspective, where it’s—
Jeremy: Ah, fair enough.
Corey: —huh, we’re taking a petabyte in and moving 300 petabytes between availability zones. It’s great. It’s a fun game called find whatever is chatty because, on some level, it’s like, run two of whatever that is—or three—rather than having it replicate. What is the deal here? And just try to identify, especially in the godforsaken hellscape that is Kubernetes, what is that thing that’s talking? And sometimes flow logs are the only real tool you’ve got, other than oral freaking tradition.
Jeremy: But God forbid you forgot to tag your [ENI 00:24:53] so that the flow log can actually be attributed to, you know, what workload is responsible for it behind the scenes. And so yeah, I mean, I think that’s a—boy that’s a case study and, like, a miserable job that I don’t think anybody would really want to have in this day and age.
Corey: The timing of this is apt. I sent out my newsletter for the week a couple hours before this recording, and in the bottom section, I asked anyone who’s got an interesting solution for solving what’s talking to what with VPC flow logs, please let me know because I found this original thing that AWS put up as part of their workshops and a lab to figure this out, but other than that, it’s more or less guess-and-check. What is the hotness? It’s been a while since I explored the landscape. And now we see if the audience is helpful or disappoints me. It’s all on you folks.
Jeremy: Isn’t the hotness to segregate every microservice into an account and run it through a load balancer so that it’s like much more properly tagged and it’s also consumable on an account-by-account basis for better attribution?
Corey: And then everything you see winds up incurring a direct fee when passing through that load balancer, instead of the same thing within the same subnet being able to talk to one another for free.
Jeremy: Yeah, yeah.
Corey: So, at scale—so yes, for visibility, you’re absolutely right. From a, I would like to spend less money giving it directly to Amazon, not so much.
Jeremy: [unintelligible 00:26:08] spend more money for the joy of attribution of workload?
Corey: Not to mention as well that coming into an environment that exists and is scaled out—which is sort of a prerequisite for me going in on a consulting project—and saying, “Oh, you should rebuild everything using serverless and microservice principles,” is a great way to get thrown out of the engagement in the first 20 minutes. Because yes, in theory, anyone can design something great, that works, that solves a problem on a whiteboard, but most of us don’t get to throw the old thing away and build fresh. And when we do great, I’m greenfielding something; there’s always constraints and challenges down the road that you don’t see coming. So, you finally wind up building the most extensible thing in the universe that can handle all these things, and your business dies before you get to MVP because that takes time, energy and effort. There are many more companies that have died due to failure to find product-market fit than have died because, “Oh hey, your software architecture was terrible.” If you hit the market correctly, there is budget to fix these things down the road, whereas your code could be pristine and your company’s still dead.
Jeremy: Yeah. I don’t really have a solution for you on that one, Corey [laugh].
Corey: [laugh].
Jeremy: I will come back to your one question—
Corey: I was hoping you did.
Jeremy: Yeah, sorry. I will come back to the question about, you know, how should people kind of get started in thinking about assessing security. And you know, to your point, look, I mean, I think Config is a low-ish cost, but should it cost anything? Probably not, at least for, like, basic CIS foundation benchmark checks. I mean, like, if the best practice that Amazon tells everybody is, “Turn on these 40-ish checks at last count,” you know, maybe those 40-ish checks should just be free and included and on in everybody’s account for any account that you tag as production, right?
Like, I will wholeheartedly agree with that sentiment, and it would be a trivial thing for Amazon to do, with one kind of caveat—and this is something that I think a lot of people don’t necessarily understand—collecting all the required data for security is actually really expensive. Security is an extremely data-intensive thing at this day and age. And I have a former coworker who used to hate the expression that security is data science, but there is some truth in it at this point, other than the kind of the magic around it is not actually that big because there’s not a lot of, let’s say, heuristic analysis or magic that goes into what queries, et cetera. A lot of security is very rule-based. It’s a lot of, you know, just binary checks: is this bit set to zero or one?
And some of those things are like relatively simple, but what ends up inevitably happening is that customers want more out of it. They don’t just want to know, is my security good or bad? They want to know things like is it good or bad now relative to last week? Has it gotten better or worse over time? And so, then you start accumulating lots of data and time series data, and that becomes really expensive.
And secondarily, the thing that’s really starting to happen more and more in the security world is correlation of multiple layers of data, infrastructure with applications, infrastructure with operating system, infrastructure with OS and app vulnerabilities, infrastructure plus vulnerabilities plus Kubernetes configurations plus API sitting at the edge of that. Because realistically, like, so many organizations that are built out at scale, the truth of the matter is, is just like on their operating system vulnerabilities, they’re going to have tens of thousands, if not millions of individual items to deal with and no human can realistically prioritize those without some context around it. And that is where the data, kind of, management becomes really expensive.
Corey: I hear you. Particularly the complaints about AWS Config, which many things like Control Tower setup for you. And on some level, it is a tax on using the cloud as the cloud should be used because it charges for evaluation of changes to your environment. So, if you’re spinning things up all the time and then turning them down when they’re not in use, that incurs a bunch of Config charges, whereas if you’ve treat it like a big dumb version of your data center where you just spin [unintelligible 00:30:13] things forever, your Config charge is nice and low. When you start seeing it entering the top ten of your spend on services, something is very wrong somewhere.
Jeremy: Yeah. I would actually say, like, a good compromise in my mind would be that we should be included with something like business support. If you pay for support with AWS, why not include Config, or some level of Config, for all the accounts that are in scope for your production support? That would seem like a very reasonable compromise.
Corey: For a lot of folks that have it enabled but they don’t see any direct value from it either, so it’s one of those things where not knowing how to turn it off becomes a tax on what you’re doing, in some cases. In SCPs, but often with Control Tower don’t allow you to do that. So, it’s your training people who are learning this in their test environments to avoid it, but you want them to be using it at scale in an enterprise environment. So, I agree with you, there has to be a better way to deliver that value to customers. Because, yeah, this thing is now, you know, 3 or 4% of your cloud bill, it’s not adding that much value, folks.
Jeremy: Yeah, one thing I will say just on that point, and, like, it’s a super small semantic nitpick that I have, I hate when people talk about security as a tax because I think it tends to kind of engender the wrong types of relationships to security. Because if you think about taxes, two things about them, I mean, one is that they’re kind of prescribed for you, and so in some sense, this kind of Control Tower implementation is similar because, like you know, it’s hard for you to turn off, et cetera, but on the other hand, like, you don’t get to choose how that tax money is spent. And really, like, you get to set your security budget as an organization. Maybe this Control Tower Config scenario is a slight outlier on that side, but you know, there are ways to turn it off, et cetera.
The other thing, though, is that, like, people tend to relate to tax, like, this thing that they really, really hate. It comes once a year, you should really do everything you can to minimize it and to, like, not spend any time on it or on getting it right. And in fact, like, there’s a lot of people who kind of like to cheat on taxes, right? And so, like, you don’t really want people to have that kind of mindset of, like, pay as little as possible, spend as little time as possible, and yes, let’s cheat on it. Like, that’s not how I hope people are addressing security in their cloud environments.
Corey: I agree wholeheartedly, but if you have a service like Config, for example—that’s what we’re talking about—and it isn’t adding value to you, and you just you don’t know what it does, how it works, than it [unintelligible 00:32:37]—or more or less how to turn it off, then it does effectively become directly in line of a tax, regardless of how people want to view the principle of taxation. It’s a—yeah, security should not be a tax. I agree with you wholeheartedly. The problem is, is it is—
Jeremy: It should be an enabler.
Corey: —unclea—yeah, the relationship between Config and security in many cases is fairly attenuated in a lot of people’s minds.
Jeremy: Yeah. I mean, I think if you don’t have, kind of, ideas in mind for how you want to use it or consume it, or how you want to use it, let’s say as an assessment against your own environment, then it’s particularly vexing. So, if you don’t know, like, “Hey, I’m going to use Config. I’m going to use Config for this set of rules. This is how I’m going to consume that data and how I’m going to then, like, pass the results on to people to make change in the organization,” then it’s particularly useless.
Corey: Yeah. I really want to thank you for taking the time to speak with me. If people want to learn more, where’s the best place for them to find you?
Jeremy: Easy, breezy. We are just firetail.io. That’s ‘fire’ like the, you know, flaming substance, and ‘tail’ like the tail of an animal, not like a story. But yeah, just firetail.io.
And if you come now, we’ve actually got, like, a white paper that we just put out around API security and kind of analyzing ten years of API-based data breaches and trying to understand what actually went wrong in most of those cases. And you’re more than welcome to grab that off of our website. And if you have any questions, just reach out to me. I’m just jeremy@firetail.io.
Corey: And we’ll put links to all of that in the [show notes 00:34:03]. Thank you so much for your time. I appreciate it.
Jeremy: My pleasure, Corey. Thanks so much for having me.
Corey: Jeremy Snyder, founder and CEO at Firetail. I’m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment pointing out that listening to my nonsense is a tax on you going about your day.
Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.