Subscribe
Copied to clipboard
Share
Share
Copied to clipboard
Embed
Copied to clipboard
Serverless Chats
Trailer
Bonus
Episode 76
Season 1
Episode #76: Building Well-Architected Serverless using CDK Patterns with Matt Coulter
About Matt Coulter
Matt Coulter is a Technical Architect at Liberty IT and AWS Community Builder. Matt has a proven history of delivering scalable, serverless solutions on the public cloud, and has crafted CDK Patterns, an open source collection of AWS Serverless architecture patterns built with CDK for developers to use. In addition to his work on CDK Patterns, he shares his passion and knowledge on serverless through events like AWS Community Day Dublin and blog posts on Dev.to.
Website: www.mattcoulter.com
Twitter: twitter.com/NIDeveloper
CDK Day: www.cdkday.com
CDK Patterns: www.cdkpatterns.com
Watch this episode on YouTube: https://youtu.be/wKvaCsvfJ_M
Transcript
Jeremy: Hi, everyone. I’m Jeremy Daly and this is Serverless Chats. Today I’m joined by Matt Coulter. Hey, Matt, thanks for joining me.
Matt: Hey, Jeremy, thanks for having me on today. I’m looking forward to this discussion so much.
Jeremy: Awesome. So you are a technical architect at Liberty IT so why don’t you tell the listeners a little bit about your background and what you do at Liberty IT.
Matt: Sure. So I’m in this account enabling architect role now at Liberty IT. What that really means is Liberty is global, it’s huge, so Liberty IT in Belfast and Dublin has about two hundred engineers in my section, but globally there’s over a thousand engineers. And if you Google Liberty Mutual serverless you can see that we have a mission, we have a mandate, we want to be a serverless first company rapidly delivering value in a well-architected way. So my job is to create the environment where our engineers can do that at a global scale, so not just one thing, but do it in a way where we don’t leave everybody behind and everybody feels bought-in and a part of doing that job.
Jeremy: Right. And if anybody has been paying attention on Twitter or is anywhere near I would say the CDK space they’ve probably come across CDKpatterns.com which is a site that you put together and I want to talk about that because that is super-interesting just in and of itself. But there’s also ... part of the reason you built this, and we’ll get into this, but is because the CDK is so powerful. And I’m going to do a little mea culpa here. At the beginning of 2020, I was looking at the CDK as, like, “I don’t know. I like DSLs better than this idea of imperative code for infrastructures code.” I think I have completely changed my mind on this just because of how powerful CDK is, especially encapsulating functionality for teams. So, I want to talk about the CDK first then we’ll get into CDK patterns. So, let’s start there; let’s start with the CDK and in case people are not familiar with what CDK is, can you explain that and give us some of the vocabulary that new listeners might need to know in order for us to have this conversation so they can follow along.
Matt: Absolutely. So, the first thing is, and I learned this just for CDK Day, CDK itself, which stands for the Cloud Development Kit, is actually not a family of products. So, if you just say “CDK” you’re actually referring to AWS CDK which is the original, the main kit, that is used to deploy resources on the AWS using Python, Java, TypeScript, pretty much they’re working on a lot of languages but there’s also CDK for Terraform which has come up through the community and it’s an officially supported product, CDK for Kubernetes, and I saw CDK for Azure. So, what brings those things together as umbrella products is is a thing called Construct, and Construct is an open source product and that is the magic behind the CDK. That is the thing that allows you to write code in your normal language and it gets converted into DSL that was the original thing that was used in the first place.
So, we’re probably going to spend this conversation talking about AWS CDK and what it does is it converts all of your code into cloud formation and that’s the brilliance of CDK. And pairing developers with the languages they know but at the end of the day you can still apply your rigor and compliance and cloud formation knowledge to the full set-up. And just one more term that might come up later, whenever we talk about Construct, we talk about L1, L2, and L3 constructs. So it’s simple-ish to remember: L1 means cloud formation, everything is L1 starts with c and fn. L2 means AWS built it, so that’s their very light opinion on how to make things easier. And then L3 is stuff that we built, that is typically an aggregation of multiple L2s. I think that’s pretty much the vocabulary you need to understand this discussion anyway.
Jeremy: All right, well, that’s good. It’s a good place for us to start. And I think this is why things have changed my mind because of that L3 category there, right, and the L3 constructs, and it’s because what has happened is rather than you just defining your infrastructure using code, whatever, TypeScript or Java or whatever, rather than you just defining it that way, constructs are multiple pieces of infrastructure that can be wrapped up together especially when you start building these level 3 ones and that allows you to wrap up all your compliance, all of your observability, and all of your metrics, all of your alarms, everything wrapped into one. And so you can do similar things with serverless or with … even with SAM, so why did you choose to do the CDK at Liberty Mutual if it’s kind of possible to do some of these other things, I know you have to copy and paste a lot, but why is the CDK so much more powerful? Why did you choose that at Liberty?
Matt: Yeah, so, I’ll tell you a story. So, a while ago, a few years ago, I had this awesome team. And we were known as a team that built a suite of microservices to support the insurance app, so multi-billion dollar apps were reliant on our services. And that meant we were specialists in Spring Boot and as well as Python for deploying machine learning models and Docker. And so there were five of us on the team, I think, and we were supporting and maintaining roughly thirty microservices. Now this was a high-performing team but I spent way too much of my time talking to our business partners saying we need to do ops. It was a case of, “Okay we need to upgrade this service from Spring Boot 1.x to a higher version or … you know, I was talking to our senior architect at the time who challenged me on a project we called “Deploy with Confidence” and he had said, “I want you to tell me if the system breaks before a customer calls you.” So, it was that point in time I had this brilliant idea of, “You know what, guys? Let’s go serverless.”
And I remember calling the team into the room and I said, “It won’t be that bad; just put in a couple lines of code in the Lambda function and we’re good. We can get rid of all the Spring Boot, we can get rid of all the frameworks, it’ll be easy.” It was not easy. It was challenging to say the least. Because our first piece of code that we put out there was an API gateway and one Lambda function and nothing else. And that Lambda function just made a call to a third-party API. That took us months to get working and that was because whenever you’re deploying code into our cloud in particular, it’s very locked-down. We have these tools in place that if you try and deploy cloud formation that isn’t up to our standards, it just gets deleted, it’s just gone. On top of that you need to know what various AWS components you’re allowed to use. So, as you know, AWS offers options for everything.
Jeremy: Right.
Matt: You need to know which options are right for you. So, we had the use at the time of a private API gateway which I don’t know if very many people use private gateways, but we had these private API gateways with a custom authorizer Lambda and by the time we got that code written there were a thousand lines of cloud formation template. And then the team got in the discussions of, okay, we were a trunk-based development team so if we have four or five developers all working on one cloud formation template, it was just chaos, it was carnage. Because somebody delivers something but it’s not quite ready for production yet. We were not used to that. We had all our Java habits down. So, we started pulling the Lambda functions out of that main template and putting them elsewhere and that’s where we had all these different cloud … we started doing the single function cloud formation template.
Jeremy: Wow, yeah.
Matt: It was at that point we discovered that if you weren’t using aliases you had to redeploy the gateway stack even though the gateway stack was completely separate to the Lambda. It didn’t pick up changes automatically. So, we were going through this evolution of it’s not really cloud formation’s fault, but we were learning all these things about serverless along the way. And by the time we got it all done, we were really proud of what we’d built but as I said Liberty’s huge so the amount of teams who had the same idea as me and thought, “Yes, I want to do that.” And then I wrote this five-part blog series that was I don’t know how many thousands of words but it was long but it was practically a book on how to deploy an API gateway and that was in cloud formation. So, whenever I started looking at the other products like serverless framework, and SAM didn’t exist whenever I was looking at it, but you still had to override parts of the underlying cloud formation to make it compliant in our environment so for me the advantage was that it was easier to stick with the pure cloud formation because I needed to know it anyway so what was the point?
The tipping point for me was whenever I tried CDK I was able to take that same API gateway that caused us so much pain and I made a construct for it that in fourteen lines of code any developer could just literally go “new gateway that’s secured at the endpoint on the function” and that has been deployed thousands of times in the past year alone just because the developer experience was what you would expect. And the beauty of it was the other abstractions for me are they are pure abstractions so it’s quite hard to kick the tires, so to speak, and understand what it’s trying to do. But because CDK is cloud formation I was able to do CDK synth to a cloud formation template itself and because I knew the cloud formation I knew everything it was doing and I knew it was good and I could pipe that into SAM and to pair the two products and start up the API gateway locally. So I was able to create this compelling vision to the engineers to say, “Here’s something like what you had with Spring Boot before, short amount of code, you can start it locally, oh, by the way, you can write infrastructure unit tests for this as well so you can do your CI/CD pipelines.
Jeremy: Yeah.
Matt: That’s why … because it brought all those skills that we already had and it transformed the developer experience into pretty much what we expected in the first place.
Jeremy: I mean, that’s amazing. Just the … when you did that putting all that out there saying we’re going to completely change the way that Liberty IT builds applications. It’s sort of like a major sort of career, like betting on your career in a way, right?
Matt: Yeah. I mean, it’s funny, because as you said yourself at the start of 2020 I went out there loud and proud talking about you know what, I think CDK will work and not only will it work, it will work for serverless. And pretty much everyone looked at me going, what has Matt been drinking today? I don’t think he’s right. But I’ve stuck with it and I think the key has been sticking to open source and talking about this stuff publicly rather than if I’d just done everything in Liberty. We’d be having a conversation where you’d be saying I still don’t know what CDK can do.
Jeremy: Right. And I want to get into the CDK patterns site that you did but this is probably a good place to bring this in and then we can go back to sort of what happened within Liberty. So, you’ve got thousands of engineers, you've hundreds of teams, or hundreds of engineers, it’s a very big company, the Liberty IT piece of it. So, you’ve got all these engineers, you’ve got tons of different teams, so you come up and write these few constructs, you start coming up with these ideas showing people how this can work, but how do you then get … that’s in one or two teams, that’s a small pod within that organization. How do you get an entire company, especially an enterprise to adopt that standard?
Matt: Yeah. So, it helps that Liberty Mutual as a whole is split up into different business segments, so my segment, GRS we call it, Global Risk Solutions, I’m lucky I remembered that, we’re basically large commercial and specialist insurance. But our CIO made a mandate; he put down what our vision is as a company and where we want to go, and he wrote down that we want to be a serverless first company. So whenever you have buy-in at the executive level, it helps a long way. But the second part of it is I haven’t mandated anything to any engineer who works anywhere because I’ve seen an awful lot of times that it doesn’t matter how good your idea is, if you come in and tell people, “I think I know better than you,” they just say no.
So that’s why I started with CDK patterns external, which is, given I haven’t introduced it yet, an open source collection of serverless architecture patterns and the idea was if I could go external and say, “Here is a thing, here is an actual industry thing, here are all the AWS Heroes that talk about the patterns that are in this, here’s the links to all their blogs posts, here are all their articles, here is me talking about it in the world and then go to them and conduct a well-architected review with their team and then instead of mandating it, just ask them, “Okay, I see you’re trying to build this particular solution, have you considered.” And then at that point because the things already exists, it’s already coded and they can pick up on it, I think you’ve reduced the barrier from the direction you want them to go rather than forcing it.
Jeremy: Right. Yeah, and I know, again, brilliant idea to get community support, right, and then you get external pressure kind of pushing down on the organization because that is what the community is using, it’s sort of community accepted. And the other thing that’s great, and why I love open source, is community review. Right? You put something out there and you say, “Here’s this pattern for doing x, y, z,” and you have people coming back saying, “Well, it would be better if you did this this way, or here’s an alternate way to do it” or whatever, then it just makes it so much … it makes everything better. Everyone gets better because of that.
So you mentioned “well-architected” and I know this is a big thing in your organization. But also more broadly in the serverless community where standards, best practices, again, patterns, what are the best patterns to use, what are some of the pitfalls, what are some of the workarounds, that is an ongoing challenge in the serverless world right now. Enforcing those standards and enforcing the compliance or the frameworks, or I guess, the well-architected framework within your organization, why is that such a huge priority for you and how do the CDK patterns help with that?
Matt: Yeah, so well-architected … When I first looked at that … Well, I’ll break it down for people if you haven’t been following. So, there’s the well-architected white paper, which is several pages of just AWS thinks you should be building anything on their cloud. And then there’s the well-architected tool, which is in the console that lets you answer a bunch of questions and it gives you advice on where they think you should go based on your workload. But they’ve still been refining it for others. So after that, AWS released a bunch of specific lenses and one of those lenses is the serverless lens of the well-architected framework and that’s a bunch of serverless-focused questions about your architecture based on the pillars of the well-architected framework. And the reason why it’s been so transformative for us is because, again, we haven’t introduced it as a pass or fail function. It’s a mechanism we use to have a conversation with our engineers.
So, we do these things, I’ll say we do it once every three months with it with a theme for the purpose if sitting down and saying, “This is the spec for what AWS thinks, again, not my opinion, this is not the Matt Coulter opinion of what you should be building. This is Heitor Lessa and the brilliant minds at AWS have said this is the serverless lens, so can you tell me … I can see that your solution will scale, but do you want it to scale that much? You know, are you going to cause a problem elsewhere or how do you know if a piece is broken and I just, I think the fact that that’s there and it’s broken down based on those pillars and it’s not my opinion has helped us massively just have a base understanding across the order that it doesn’t matter what business unit you’re in or even which technology you;re coding in, this is universal. It’s been a massive help in that way.
Jeremy: Yeah. And so that’s one of the things I think that, again, going back to the CDK is why I’ve changed my mind on it so much is that you can use these constructs to encapsulate some of these best practices into it. I mean in terms of the actual pillars, you know like just being able to have observability and some of these other things, security and all that stuff, and then again being such a large enterprise, I’m sure there are a lot of lawyers that work at Liberty IT and Liberty Mutual to make sure that all these things get passed and all these things follow proper compliance and then it’s just every other major compliance thing that’s out there, whether it’s PCI or SOC 2 or whatever those things are, all of that stuff, any bit of it that can be encapsulated into these constructs, is just there. I think that’s amazing. Okay, so let’s move on a little bit and get more specific about how you implement the CDK because clearly you’ve got the CDK patterns on the outside, you know, sort of that pressure coming in, I know you use those patterns internally as well. But what about in a large organization, I think about something like dependency management, right, so how do you handle, I mean, you must have shared components across teams and things like that, so how do you do dependency management in CDK at Liberty IT?
Matt: Yeah, so it’s something I haven’t touched upon yet. So, CDK patterns as I’ve said, is the external facing, open source collection of patterns and you use them internally. Well, that is true. We use them internally in the sense that there’s a Liberty Mutual tailored version of every pattern but we also have a tool called the software accelerator. And what that is is essentially, “click, click, click” new pipeline set up, new code base set up, pattern deployed, so it’s a rapid tool for developers getting up to speed. The reason why that’s relevant to this question is because it means say that gateway that I just mentioned, that is a custom construct that we have in npm but the accelerator it just pulls you in a version of … it just pulls in, say version 3, but it’s still npm and you can update it any time. So we handle our dependency management based on standard practices based on your releases so, we’ve been looking at it today as … there’s a big discussion about this in the community for CDK with should you release a new version of your construct with every new version of CDK or should you build it in a way where you say any version above this point is good with this construct.
There’s pros and cons for both ways. The reason why you release it with every version is because you can guarantee you’ve tested it, it works, you’ve pulled in the latest version of the dependencies. But the only downside is you need to release a new version for every version of CDK which happens maybe twice a week, so if you don’t have that automated it’s a lot of work. The other way is you could say my construct works with anything above version, say 1.30, and that’ll work for the most part until you get a breaking API change and then you have to decide am I going to have to release a new version and communicate to people to work about this version you will need to update and the shorter end of that method of the consumer tells you that the construct’s broken before you know yourself unless you’re testing every version anyway. So that’s why we use locked versions and we do updates with every version of CDK entirely but that works well because, as I said, everybody is working off those base patterns so it’s not like there’s 5,000 different things to update whenever the API gateway construct gets a new version, it’s just a case we have a themes channel, just update the themes channel and everybody knows a new version is out there and they can update.
Jeremy: Now, not to get too deep into the weeds, but that’s changing with v2 of the CDK, right?
Matt: Yeah. So, at the minute, originally whenever CDK was launched, they thought it would be good for all those L2s that AWS made, the opinionated constructs the very light opinionated ones, they released them all independently, but the problem was that they all need to be on the same version of CDK today. So, say you start building your project and it’s 1.70 and then you decide to add a new dependency but you just do npm install and 1.71 has been released since then, well, the new thing you just pulled in would get 1.71 and you would get a weird … it doesn’t actually say your dependencies mismatch, you get a weird TypeScript error and the way it affects it is to make sure that all your dependencies are in the same version. But what they’re doing with v2, which I don’t know the exact release date yet, but we keep saying it will be a few months out, is they’re model CDK, all those AWS L2s are going to be bundled inside CDK’s core module. So it means that for third-party ones you might have that issue slightly but all the AWS ones at least will be on the same version because they’re bundled together.
Jeremy: Awesome. Okay now, so what about testing. So you mentioned that you can do some automated testing and sort of build that into the CI/CD to test your cloud formation. So how do you implement testing on the CDK stuff?
Matt: Yeah, so this is something I particularly like about CDK. It’s better in the TypeScript version than in the other languages, unfortunately. There are plans as far as I’m aware to deport the testing for the other languages, it just hasn’t been done yet. But if you look at all the patterns on the CDKpatterns.com they all have tests. And there’s multiple different levels of tests you can do. So at the most basic level you can do a snapshot which is to say, “This cloud formation should not change and if it does change my pipeline should break; I don’t want this to deploy.” And for me, that’s the kind of test you put in after your application is stable. You’re not adding any new features but you’re upgrading CDK versions and you just want to make sure nothing breaks. On top of that, you can actually do some more complicated granular tests. So you can go in to write a unit test to say “I expect this cloud formation to have a resource like a Lambda function with this particular handler. And I tend to write a suite of tests all based around that so that that way I can at least know without having to say “this whole cloud formation is the same,” the very important bits on a unit test run I can check that they’re all there. I think that’s awesome.
Jeremy: That’s, again, I mean, just writing tests … that’s one of the things that is so tough with you know the DSLs if you just have cloud formation running the tests on just cloud formation is not very easy to do. So, if you have constructs that are generating then you can run that, that’s a very good approach to doing that.
Matt: We’ve all been there with cloud formation where, as I said, there was a team of four or five of us and somebody would change something that they thought was minor and all of a sudden the cloud formation doesn’t deploy and then we’re all sitting around going “what changed?” So, yeah, that’s why I love the unit test because you know the unit tests are on the cloud formation itself so they’re not on the Java code or the TypeScript code, they’re on the cloud formation. So you know if they pass at least you’ve a valid cloud formation template that should theoretically deploy which is nine tenths of the battle.
Jeremy: Right, right, absolutely. All right so let’s move on to the CDK pattern site. First of all, let’s take a step back. You explained it: it’s a collection of patterns and constructs for CDK that basically outline a number of things. What are there, twenty-three patterns now?
Matt: Twenty-three today, yeah.
Jeremy: Twenty-three patterns. So all these really great patterns, everything from webhooks to cloud formation to Alexa skills. All these other things. So, we kind of got the background of why you build it but really what was really the biggest motivation, what was the trigger point that said I’m going to go out there and spend all of my free time here building this amazing site that people can use.
Matt: Yeah, so I got all this CDK, let’s say it was probably last July or August, was whenever I really got into CDK and at that point I did not think I was going to start going to open source for coding. But it was about the start of November I launched that API gateway pattern internally. I was so confident. I was like, “this is it, we’re done—we’ve API gateway and Lambda, we are good.” But I got to go to re:Invent last year, I was so lucky to get one of the tickets. And I realized that most of the problem was actually just the sheer amount of options for developers and what they can do. You know, it doesn’t matter who you follow, if it’s yourself or any of the advocates or who, there’s a million and one different patterns and opinions for how you can build these things so I was sitting there and I thought how can I help this situation, because as I mentioned earlier, you get a pattern, say I took the simple web service from yourself which is just API gateway Lambda DynomoDb but if I try to deploy that internally I personally know all of the extra steps I have to go to deploy that at Liberty Mutual, but how could I help all the other engineers know that.
So that’s why I had this moment of decision where I could either do it internally or I could go externally and go open source. And there’s a tweet out there from I think January where I said, “Sod it. Be the change you want to see in the world.” I bought CDKpatterns.com and let’s get this created. And ever since then, as you say, I’ve had no free time, pretty much every free second just reading and coding. But the reason why I’ve kept doing it is because the first couple of patterns launched and I said I was doing it because I wanted the other engineers to know what it is, how they can deploy things, and what they need to configure and it really internally did hit the mark and massively helped. So never mind the fact that externally people have picked up on it and it has made my life so much easier as an architect being able to use it as the base conversation, even the likes of this where we know we have the same language to have these conversations.
Jeremy: Yeah, that’s amazing. So let’s get into CDK Patterns itself. So, first of all, CDKpatterns.com. Go and check it out. It’s not just a list of patterns, it is organized around the well-architected framework as well, the serverless lens of that. So just explain the organization so people get that.
Matt: Yup. So there’s a few different ways you can find patterns on the site. Originally it was just a GitHub repo, and I’ve sort of been agiling my way to a product here. So the first way is you can just view all the patterns. You can just go in and click “view all” and you can just see all the pictures and scroll through them. Or you can go in and view them through the serverless component used. So you could say I need a pattern for CloudFront , go ahead and pick CloudFront and it will filter them by the patterns that use CloudFront. But the most interesting way I’ve been trying to do it is, I talked to Heitor Lessa and said, “I want to try to introduce well-architected to CDK patterns” and originally I had this really like hit thing, but in talking to him it became a case of saying, if I’m conducting a well-architected review and somebody goes through that and they say question two is your problem, how do they know the answer to that question is the particular pattern. So that’s why if you go to the well-architected section of CDKpatterns.com it’s broken down by each question in the well-architected serverless lens and then each question doesn’t have one answer it has multiple best practices. So underneath each question there’s a best practice and if I have a pattern that helps with that best practice it’s right there, if I don’t it links to all the AWS docs so you should at least be able to find something.
Jeremy: Awesome. Let’s talk about some of these patterns. So like we said, there are twenty-three of them right now. So what are some of your favorite ones?
Matt: Yeah. The one that took the most effort, which I don’t know if people realize looking at it, was the CloudWatch dashboard pattern.
Jeremy: Right. This is a good pattern.
Matt: That took me probably the longest of all the patterns, I’d say three and a half weeks of just reading theory. So it’s not so much the implementation but what to implement was the problem there, and that goes back to it’s taking the simple web service that I mentioned earlier, API gateway Lambda DynomoDb but if you want to build a CloudWatch dashboard for that what are the right graphs, what are the right metrics, what are the right alerts that you’d want the test to tell you. So I had to get into metric max and you know, try to work out what do I want to alert on and how frequently. So we’ve used that pattern now internally and I personally it’s one of my favorites because it seems like it’s low-hanging fruit but there’s actually a lot more behind it once you scratch the surface.
Jeremy: Right. And I mean, there was just a tall the other day at Serverless Days Virtual from the team at Lego and one of their audit processes was figuring out the observability piece and what metrics they wanted to track and what they wanted to alert on and make sure that everything was with that. That’s a really interesting benefit of the CDK is to say, “Look, here’s the pattern that I’m launching, the connection of components that I’m launching,” but then everything else that’s around that. Just the CloudWatch metrics alone are huge. So, again, super-cool pattern for that particular implementation but a really good framework to use if you’re building out your own constructs for your own company in terms of what other patterns you might need there or what other alerts and metrics surrounding that.
Matt: Yeah, I mean, if you jump into the pattern you’ll see the list of external references are quite long in that pattern and that’s just because there’s a lot of opinions about what the right metrics are but I will say you are going to have to write your own metrics just because the ones out of the box they’re not really good enough today to just use the pure metrics. That’s why you have to get into metric max and fully understand why you’re alerting on what it is. So the pattern breaks it down the metrics for the gateway itself, the metrics form the Lambda function and the metrics for DynamoDb. So if you’ve any of those three it’s worth checking it out and seeing if there’s anything that can help you.
Jeremy: Right. Yeah, and you doing all that research for us is very, very helpful. All right, so another pattern on there that’s one of my favorites, and this is newer to the CDK patterns site, is the Lambda circuit breaker and you based this pattern off of something Gunner Grosch just released. He released a little script, a little npn package, to allow you to implement Lambda circuit breakers, which I’ve been talking about for, I don’t know, two and half years now or something like that. So, I love that this now is encapsulated in the CDK.
Matt: Yeah, it’s awesome. Whenever Gunner released that I sent him a DM, I was just “Okay, do you mind if I throw the CDK pattern for this,” because internally, you have no idea how many conversations I’ve had internally, about needing circuit breakers. I mean, it should be core at the point but it’s not so that’s why whenever I saw that library I thought, you know what, this is something that it’s not a huge effort for me to put out there but it’s of huge value to the community to be able to say if you’re going to integrate with something that may or may not be reliable, let’s give you a mechanism to decide what to do whenever it’s not at its reliable phase. It seems basic but it’s not.
Jeremy: Right. Yeah, and I still don’t understand why this is not built in to AWS or any of these cloud providers because if people are not familiar with the circuit breakers essentially what it is is when you are reaching out to third-party APIs or anything that, again, you could overwhelm, or could go down, you’re building in a mechanism here that says once I start getting a certain amount of failures I’m either going to back off or I’m going to take a break for a minute and then I will keep trying to see if it’s working but I’ll be that good netizen, if you want to use that word, and not overwhelm these downstream services because the worst this you can do is for a service that is responding slowly is send more traffic to it. So, yes, these are the things that it’s such a common thing. You’re always calling third-party APIs, you’re either writing data, you’re reading data. It will be interesting to see what they do with EventBridge and SAS integration and some of those things around that, but until then, Lambda circuit breakers, check them out, awesome pattern.
All right, what's another pattern that you like?
Matt: Another one I like, they’re all based around well-architected, but the Saga Step function. The Saga pattern, this has been around a lot longer than serverless, this is just a design pattern where for … it’s about managing these big long distributed processes of being able to say what happens when there’s failure when you’re midway through the process. For the example in this pattern, it’s a holiday booking, it’s a case of booking your flights and your hotel and I think your car maybe, I can’t remember if they added the car or not, or if it was one layer too many. But the idea is for every step that you take you need to have an equal but opposite undo step, so that if anything goes wrong it unravels layer by layer, so the system is in the state it was before you tried anything. It links to an awesome talk from GOTO by Caitie McCaffrey. And the Saga pattern itself it’s … if you want to build systems and you’re working across these multiple bounded contexts as opposed to one single unit, it’s something you really need to consider.
Jeremy: Yeah. Absolutely. And the Saga pattern is something, too, from a serverless standpoint, I know that Yan Cui, the Burning Monk, had put some stuff about that in the past, which is just … it’s just a really, really great pattern. If you’re coordinating multiple things, I mean. I’m a big fan of choreography, like using EventBridge and having other systems react. And in most cases that’s fine if someone is going to get a mass mailer or some sort of, maybe a coupon is going to go out if someone makes a certain number of purchases, if that fails it’s probably not the end of the world if that didn’t work or updated in your Salesforce or Marketo. But if the inventory system wasn’t updated, well then that’s a problem. We need to make sure those things are taken care of, so absolutely great patterns.
All right, one more pattern I want to call out. In the beginning of 202 I was thinking that voice control was going to be a hugely important thing. With all of these, I’m going to say … I’m going to mute … hold on, I’m going to mute my Alexa so it doesn’t respond to me. But with all of these voice controlled systems and all this voice interactivity, I think the ability for you to now do really complex tasks with your voice is pretty cool. I didn’t see as much this year as I thought I would. I thought there would be more of a progression. But maybe that was because everybody was locked at home and maybe there wasn’t enough opportunity for these things to grow. But I do think it’s a very cool thing and I think Alexa, obviously, is one of the pieces that’s driving this. You just recently released an Alexa skill, so tell us about that.
Matt: Yeah, what’s cool about this pattern is that I didn’t write it, so this was actually a community contribution from another member of Liberty Mutual’s staff, an IO community builder, Chris Plankey, but what’s awesome about this is we also have two Heroes at Liberty Mutual, Jillian Armstrong and Jillian McCann, both experts in machine learning, lex, and voice. So, since I’ve started CDK Patterns thinking how can I get voice e.i. into these patterns because it is something, it’s going to come at some point and be mainstream. The problem with lex in particular it’s not exactly in cloud formation today. You have to deploy your own custom resource to get it out there. But Alexa was apparently slightly easier, so that’s where Chris Plankey was able to take a very basic skill and you still have to navigate the fact that there is Amazon and there is Amazon Web Services so you need an Amazon developer to deploy it, but outside of that the barrier to deploying it is get your Amazon account, clone pattern deploy, and you’ve got Alexa skill which what it does is just let you say, “CDK Patterns tell me what patterns you have.” But you can adapt that to be whatever you want in your context.
Jeremy: Right, yeah, and I’ve developed a couple of Alexa skills more as a test, I’ve never put one out there for other people to use, but they are very cool. They do get kind of complicated, and you need to … and this is another thing, CDK Patterns isn’t going to help you understand all of the semantics and how you have to create the different questions and all that kind of stuff and all the slots, but it’s great for getting the basics set up and, like you said, just being able to iterate on that and make some changes.
All right, awesome. So, again, CDKpatterns.com, go check it out. There are twenty-three patterns there that will get you started immediately and probably get you to really like the CDK. So, do that. Speaking of people who really like the CDK, another thing that happened earlier this year which is crazy how fast this all happened was the CDK Day. So tell us about the CDK Day.
Matt: Yeah, so this was one day I just had this idea and I thought, “What if we just had one day, we just took the whole day, and just talked about everything CDK.” So I sent a message to a few of my friends and from there it just snowballed. I sort of thought everyone would say no, this is too much effort, but everybody was so supportive. And it was two weeks or four weeks after that message we actually put out the tweet to say CDK Day was happening and then it was twelve weeks after that that the day happened and in that time we had to build the website, get the CFP out there, pick the talks, find a host, get the screening platform up and running, do all the social media. But the community absolutely came through, I mean the numbers of people that signed up was incredible. At the very start for the keynote there was over a thousand people watching for a conference that was brand new and all of the speakers all nailed their particular talks. I couldn’t have asked for more considering it was just a random idea I had one day that snowballed into a large thing.
Jeremy: That’s amazing. And the videos of the talks are up on the CDKday.com, right, so you can go back and watch the replays?
Matt: Yeah, they’re all up on YouTube and you can find them. There’s a rewind page on CDKday.com where they’re all individually up there because something I’ve learned since I was a 5R, back-to-back-to-back of talks. To make it all easier for people to consume, I’ve sliced them all up and reuploaded them so yeah, you can find them all there broken down by speaker name and title.
Jeremy: Awesome. So what are some of the best talks that people should go check out?
Matt: One that is really awesome that’s still in beta or def preview is “CDK Pipelines.” This is the idea that with CDK not only can you use your language to deploy your infrastructure but you can use it for multi-region, multi-account set-ups, so everything in the same code. So Thorsten Hoeger, who is a Hero, he talks through the actual live demo of how to do that with CDK pipeline. It’s a really good talk if you haven’t seen CDK pipelines before, you’ll learn a lot.
There’s a couple of other good ones as well. So, Nader Dabit actually gives a talk on AppSync with CDK. I’m a big fan of trying to get Amplify and CDK to be friends. I’m trying to get this rapid development closer together, but he does a live demo of AppSync which is really good. And then the last one was Elad, who came up with the idea for CDK. He showed off his new project which is called projen. This is like an opinionated way of creating new projects. So you can just go in be like, projen, use CDK module, and it’s like … it abstracts certain things like all of the different dependency versions I told you, it abstracts them into a yaml for you, and you don’t have to manage that anymore. You can just give it your GitHub token and it will keep all your projects up to date and stuff. So it’s a really cool concept.
Jeremy: Awesome. All right. So listen, CDKday.com, CDKpatterens.com. So, Matt, thank you so much for, one, joining me here today, but also for literally the tons of work you have done. I don’t think you quite realize how much good you have done, not just within Liberty Mutual but in the community itself. I mean, there are … the CDK Day, everything you have been doing is amazing. Keep it up and … yeah, just awesome. If people want to find out more about you and all these other projects you’re working on and maybe what you’re taking so that you don’t need to sleep so you can keep working on all of these things, how do they do that?
Matt: Yeah, thank you for that. If you want to find me I’m @nideveloper on Twitter. To be fair, if you type in nideveloper into Google, you’ll find a ton of resources that I’m at. But if you want to keep up to date on the patterns, there’s a CDK patterns Twitter handle. If you want to stay up to date on CDK Day, in case we throw another one, there’s a CDK Day Twitter handle that you can follow. If you want blog posts, you can go to dev.2/nideveloper. That has your bases covered. If you can find me in those places, you can find the others.
Jeremy: Awesome. Well, I will put all of this stuff in the show notes, LinkedIn, the blog, your Twitter, CDK Patterns, CDK Day. Matt, thanks again, really appreciate it.
Matt: Thank you for having me.
This episode is sponsored by TriggerMesh and Amazon Web Services.
Matt Coulter is a Technical Architect at Liberty IT and AWS Community Builder. Matt has a proven history of delivering scalable, serverless solutions on the public cloud, and has crafted CDK Patterns, an open source collection of AWS Serverless architecture patterns built with CDK for developers to use. In addition to his work on CDK Patterns, he shares his passion and knowledge on serverless through events like AWS Community Day Dublin and blog posts on Dev.to.
Website: www.mattcoulter.com
Twitter: twitter.com/NIDeveloper
CDK Day: www.cdkday.com
CDK Patterns: www.cdkpatterns.com
Watch this episode on YouTube: https://youtu.be/wKvaCsvfJ_M
Transcript
Jeremy: Hi, everyone. I’m Jeremy Daly and this is Serverless Chats. Today I’m joined by Matt Coulter. Hey, Matt, thanks for joining me.
Matt: Hey, Jeremy, thanks for having me on today. I’m looking forward to this discussion so much.
Jeremy: Awesome. So you are a technical architect at Liberty IT so why don’t you tell the listeners a little bit about your background and what you do at Liberty IT.
Matt: Sure. So I’m in this account enabling architect role now at Liberty IT. What that really means is Liberty is global, it’s huge, so Liberty IT in Belfast and Dublin has about two hundred engineers in my section, but globally there’s over a thousand engineers. And if you Google Liberty Mutual serverless you can see that we have a mission, we have a mandate, we want to be a serverless first company rapidly delivering value in a well-architected way. So my job is to create the environment where our engineers can do that at a global scale, so not just one thing, but do it in a way where we don’t leave everybody behind and everybody feels bought-in and a part of doing that job.
Jeremy: Right. And if anybody has been paying attention on Twitter or is anywhere near I would say the CDK space they’ve probably come across CDKpatterns.com which is a site that you put together and I want to talk about that because that is super-interesting just in and of itself. But there’s also ... part of the reason you built this, and we’ll get into this, but is because the CDK is so powerful. And I’m going to do a little mea culpa here. At the beginning of 2020, I was looking at the CDK as, like, “I don’t know. I like DSLs better than this idea of imperative code for infrastructures code.” I think I have completely changed my mind on this just because of how powerful CDK is, especially encapsulating functionality for teams. So, I want to talk about the CDK first then we’ll get into CDK patterns. So, let’s start there; let’s start with the CDK and in case people are not familiar with what CDK is, can you explain that and give us some of the vocabulary that new listeners might need to know in order for us to have this conversation so they can follow along.
Matt: Absolutely. So, the first thing is, and I learned this just for CDK Day, CDK itself, which stands for the Cloud Development Kit, is actually not a family of products. So, if you just say “CDK” you’re actually referring to AWS CDK which is the original, the main kit, that is used to deploy resources on the AWS using Python, Java, TypeScript, pretty much they’re working on a lot of languages but there’s also CDK for Terraform which has come up through the community and it’s an officially supported product, CDK for Kubernetes, and I saw CDK for Azure. So, what brings those things together as umbrella products is is a thing called Construct, and Construct is an open source product and that is the magic behind the CDK. That is the thing that allows you to write code in your normal language and it gets converted into DSL that was the original thing that was used in the first place.
So, we’re probably going to spend this conversation talking about AWS CDK and what it does is it converts all of your code into cloud formation and that’s the brilliance of CDK. And pairing developers with the languages they know but at the end of the day you can still apply your rigor and compliance and cloud formation knowledge to the full set-up. And just one more term that might come up later, whenever we talk about Construct, we talk about L1, L2, and L3 constructs. So it’s simple-ish to remember: L1 means cloud formation, everything is L1 starts with c and fn. L2 means AWS built it, so that’s their very light opinion on how to make things easier. And then L3 is stuff that we built, that is typically an aggregation of multiple L2s. I think that’s pretty much the vocabulary you need to understand this discussion anyway.
Jeremy: All right, well, that’s good. It’s a good place for us to start. And I think this is why things have changed my mind because of that L3 category there, right, and the L3 constructs, and it’s because what has happened is rather than you just defining your infrastructure using code, whatever, TypeScript or Java or whatever, rather than you just defining it that way, constructs are multiple pieces of infrastructure that can be wrapped up together especially when you start building these level 3 ones and that allows you to wrap up all your compliance, all of your observability, and all of your metrics, all of your alarms, everything wrapped into one. And so you can do similar things with serverless or with … even with SAM, so why did you choose to do the CDK at Liberty Mutual if it’s kind of possible to do some of these other things, I know you have to copy and paste a lot, but why is the CDK so much more powerful? Why did you choose that at Liberty?
Matt: Yeah, so, I’ll tell you a story. So, a while ago, a few years ago, I had this awesome team. And we were known as a team that built a suite of microservices to support the insurance app, so multi-billion dollar apps were reliant on our services. And that meant we were specialists in Spring Boot and as well as Python for deploying machine learning models and Docker. And so there were five of us on the team, I think, and we were supporting and maintaining roughly thirty microservices. Now this was a high-performing team but I spent way too much of my time talking to our business partners saying we need to do ops. It was a case of, “Okay we need to upgrade this service from Spring Boot 1.x to a higher version or … you know, I was talking to our senior architect at the time who challenged me on a project we called “Deploy with Confidence” and he had said, “I want you to tell me if the system breaks before a customer calls you.” So, it was that point in time I had this brilliant idea of, “You know what, guys? Let’s go serverless.”
And I remember calling the team into the room and I said, “It won’t be that bad; just put in a couple lines of code in the Lambda function and we’re good. We can get rid of all the Spring Boot, we can get rid of all the frameworks, it’ll be easy.” It was not easy. It was challenging to say the least. Because our first piece of code that we put out there was an API gateway and one Lambda function and nothing else. And that Lambda function just made a call to a third-party API. That took us months to get working and that was because whenever you’re deploying code into our cloud in particular, it’s very locked-down. We have these tools in place that if you try and deploy cloud formation that isn’t up to our standards, it just gets deleted, it’s just gone. On top of that you need to know what various AWS components you’re allowed to use. So, as you know, AWS offers options for everything.
Jeremy: Right.
Matt: You need to know which options are right for you. So, we had the use at the time of a private API gateway which I don’t know if very many people use private gateways, but we had these private API gateways with a custom authorizer Lambda and by the time we got that code written there were a thousand lines of cloud formation template. And then the team got in the discussions of, okay, we were a trunk-based development team so if we have four or five developers all working on one cloud formation template, it was just chaos, it was carnage. Because somebody delivers something but it’s not quite ready for production yet. We were not used to that. We had all our Java habits down. So, we started pulling the Lambda functions out of that main template and putting them elsewhere and that’s where we had all these different cloud … we started doing the single function cloud formation template.
Jeremy: Wow, yeah.
Matt: It was at that point we discovered that if you weren’t using aliases you had to redeploy the gateway stack even though the gateway stack was completely separate to the Lambda. It didn’t pick up changes automatically. So, we were going through this evolution of it’s not really cloud formation’s fault, but we were learning all these things about serverless along the way. And by the time we got it all done, we were really proud of what we’d built but as I said Liberty’s huge so the amount of teams who had the same idea as me and thought, “Yes, I want to do that.” And then I wrote this five-part blog series that was I don’t know how many thousands of words but it was long but it was practically a book on how to deploy an API gateway and that was in cloud formation. So, whenever I started looking at the other products like serverless framework, and SAM didn’t exist whenever I was looking at it, but you still had to override parts of the underlying cloud formation to make it compliant in our environment so for me the advantage was that it was easier to stick with the pure cloud formation because I needed to know it anyway so what was the point?
The tipping point for me was whenever I tried CDK I was able to take that same API gateway that caused us so much pain and I made a construct for it that in fourteen lines of code any developer could just literally go “new gateway that’s secured at the endpoint on the function” and that has been deployed thousands of times in the past year alone just because the developer experience was what you would expect. And the beauty of it was the other abstractions for me are they are pure abstractions so it’s quite hard to kick the tires, so to speak, and understand what it’s trying to do. But because CDK is cloud formation I was able to do CDK synth to a cloud formation template itself and because I knew the cloud formation I knew everything it was doing and I knew it was good and I could pipe that into SAM and to pair the two products and start up the API gateway locally. So I was able to create this compelling vision to the engineers to say, “Here’s something like what you had with Spring Boot before, short amount of code, you can start it locally, oh, by the way, you can write infrastructure unit tests for this as well so you can do your CI/CD pipelines.
Jeremy: Yeah.
Matt: That’s why … because it brought all those skills that we already had and it transformed the developer experience into pretty much what we expected in the first place.
Jeremy: I mean, that’s amazing. Just the … when you did that putting all that out there saying we’re going to completely change the way that Liberty IT builds applications. It’s sort of like a major sort of career, like betting on your career in a way, right?
Matt: Yeah. I mean, it’s funny, because as you said yourself at the start of 2020 I went out there loud and proud talking about you know what, I think CDK will work and not only will it work, it will work for serverless. And pretty much everyone looked at me going, what has Matt been drinking today? I don’t think he’s right. But I’ve stuck with it and I think the key has been sticking to open source and talking about this stuff publicly rather than if I’d just done everything in Liberty. We’d be having a conversation where you’d be saying I still don’t know what CDK can do.
Jeremy: Right. And I want to get into the CDK patterns site that you did but this is probably a good place to bring this in and then we can go back to sort of what happened within Liberty. So, you’ve got thousands of engineers, you've hundreds of teams, or hundreds of engineers, it’s a very big company, the Liberty IT piece of it. So, you’ve got all these engineers, you’ve got tons of different teams, so you come up and write these few constructs, you start coming up with these ideas showing people how this can work, but how do you then get … that’s in one or two teams, that’s a small pod within that organization. How do you get an entire company, especially an enterprise to adopt that standard?
Matt: Yeah. So, it helps that Liberty Mutual as a whole is split up into different business segments, so my segment, GRS we call it, Global Risk Solutions, I’m lucky I remembered that, we’re basically large commercial and specialist insurance. But our CIO made a mandate; he put down what our vision is as a company and where we want to go, and he wrote down that we want to be a serverless first company. So whenever you have buy-in at the executive level, it helps a long way. But the second part of it is I haven’t mandated anything to any engineer who works anywhere because I’ve seen an awful lot of times that it doesn’t matter how good your idea is, if you come in and tell people, “I think I know better than you,” they just say no.
So that’s why I started with CDK patterns external, which is, given I haven’t introduced it yet, an open source collection of serverless architecture patterns and the idea was if I could go external and say, “Here is a thing, here is an actual industry thing, here are all the AWS Heroes that talk about the patterns that are in this, here’s the links to all their blogs posts, here are all their articles, here is me talking about it in the world and then go to them and conduct a well-architected review with their team and then instead of mandating it, just ask them, “Okay, I see you’re trying to build this particular solution, have you considered.” And then at that point because the things already exists, it’s already coded and they can pick up on it, I think you’ve reduced the barrier from the direction you want them to go rather than forcing it.
Jeremy: Right. Yeah, and I know, again, brilliant idea to get community support, right, and then you get external pressure kind of pushing down on the organization because that is what the community is using, it’s sort of community accepted. And the other thing that’s great, and why I love open source, is community review. Right? You put something out there and you say, “Here’s this pattern for doing x, y, z,” and you have people coming back saying, “Well, it would be better if you did this this way, or here’s an alternate way to do it” or whatever, then it just makes it so much … it makes everything better. Everyone gets better because of that.
So you mentioned “well-architected” and I know this is a big thing in your organization. But also more broadly in the serverless community where standards, best practices, again, patterns, what are the best patterns to use, what are some of the pitfalls, what are some of the workarounds, that is an ongoing challenge in the serverless world right now. Enforcing those standards and enforcing the compliance or the frameworks, or I guess, the well-architected framework within your organization, why is that such a huge priority for you and how do the CDK patterns help with that?
Matt: Yeah, so well-architected … When I first looked at that … Well, I’ll break it down for people if you haven’t been following. So, there’s the well-architected white paper, which is several pages of just AWS thinks you should be building anything on their cloud. And then there’s the well-architected tool, which is in the console that lets you answer a bunch of questions and it gives you advice on where they think you should go based on your workload. But they’ve still been refining it for others. So after that, AWS released a bunch of specific lenses and one of those lenses is the serverless lens of the well-architected framework and that’s a bunch of serverless-focused questions about your architecture based on the pillars of the well-architected framework. And the reason why it’s been so transformative for us is because, again, we haven’t introduced it as a pass or fail function. It’s a mechanism we use to have a conversation with our engineers.
So, we do these things, I’ll say we do it once every three months with it with a theme for the purpose if sitting down and saying, “This is the spec for what AWS thinks, again, not my opinion, this is not the Matt Coulter opinion of what you should be building. This is Heitor Lessa and the brilliant minds at AWS have said this is the serverless lens, so can you tell me … I can see that your solution will scale, but do you want it to scale that much? You know, are you going to cause a problem elsewhere or how do you know if a piece is broken and I just, I think the fact that that’s there and it’s broken down based on those pillars and it’s not my opinion has helped us massively just have a base understanding across the order that it doesn’t matter what business unit you’re in or even which technology you;re coding in, this is universal. It’s been a massive help in that way.
Jeremy: Yeah. And so that’s one of the things I think that, again, going back to the CDK is why I’ve changed my mind on it so much is that you can use these constructs to encapsulate some of these best practices into it. I mean in terms of the actual pillars, you know like just being able to have observability and some of these other things, security and all that stuff, and then again being such a large enterprise, I’m sure there are a lot of lawyers that work at Liberty IT and Liberty Mutual to make sure that all these things get passed and all these things follow proper compliance and then it’s just every other major compliance thing that’s out there, whether it’s PCI or SOC 2 or whatever those things are, all of that stuff, any bit of it that can be encapsulated into these constructs, is just there. I think that’s amazing. Okay, so let’s move on a little bit and get more specific about how you implement the CDK because clearly you’ve got the CDK patterns on the outside, you know, sort of that pressure coming in, I know you use those patterns internally as well. But what about in a large organization, I think about something like dependency management, right, so how do you handle, I mean, you must have shared components across teams and things like that, so how do you do dependency management in CDK at Liberty IT?
Matt: Yeah, so it’s something I haven’t touched upon yet. So, CDK patterns as I’ve said, is the external facing, open source collection of patterns and you use them internally. Well, that is true. We use them internally in the sense that there’s a Liberty Mutual tailored version of every pattern but we also have a tool called the software accelerator. And what that is is essentially, “click, click, click” new pipeline set up, new code base set up, pattern deployed, so it’s a rapid tool for developers getting up to speed. The reason why that’s relevant to this question is because it means say that gateway that I just mentioned, that is a custom construct that we have in npm but the accelerator it just pulls you in a version of … it just pulls in, say version 3, but it’s still npm and you can update it any time. So we handle our dependency management based on standard practices based on your releases so, we’ve been looking at it today as … there’s a big discussion about this in the community for CDK with should you release a new version of your construct with every new version of CDK or should you build it in a way where you say any version above this point is good with this construct.
There’s pros and cons for both ways. The reason why you release it with every version is because you can guarantee you’ve tested it, it works, you’ve pulled in the latest version of the dependencies. But the only downside is you need to release a new version for every version of CDK which happens maybe twice a week, so if you don’t have that automated it’s a lot of work. The other way is you could say my construct works with anything above version, say 1.30, and that’ll work for the most part until you get a breaking API change and then you have to decide am I going to have to release a new version and communicate to people to work about this version you will need to update and the shorter end of that method of the consumer tells you that the construct’s broken before you know yourself unless you’re testing every version anyway. So that’s why we use locked versions and we do updates with every version of CDK entirely but that works well because, as I said, everybody is working off those base patterns so it’s not like there’s 5,000 different things to update whenever the API gateway construct gets a new version, it’s just a case we have a themes channel, just update the themes channel and everybody knows a new version is out there and they can update.
Jeremy: Now, not to get too deep into the weeds, but that’s changing with v2 of the CDK, right?
Matt: Yeah. So, at the minute, originally whenever CDK was launched, they thought it would be good for all those L2s that AWS made, the opinionated constructs the very light opinionated ones, they released them all independently, but the problem was that they all need to be on the same version of CDK today. So, say you start building your project and it’s 1.70 and then you decide to add a new dependency but you just do npm install and 1.71 has been released since then, well, the new thing you just pulled in would get 1.71 and you would get a weird … it doesn’t actually say your dependencies mismatch, you get a weird TypeScript error and the way it affects it is to make sure that all your dependencies are in the same version. But what they’re doing with v2, which I don’t know the exact release date yet, but we keep saying it will be a few months out, is they’re model CDK, all those AWS L2s are going to be bundled inside CDK’s core module. So it means that for third-party ones you might have that issue slightly but all the AWS ones at least will be on the same version because they’re bundled together.
Jeremy: Awesome. Okay now, so what about testing. So you mentioned that you can do some automated testing and sort of build that into the CI/CD to test your cloud formation. So how do you implement testing on the CDK stuff?
Matt: Yeah, so this is something I particularly like about CDK. It’s better in the TypeScript version than in the other languages, unfortunately. There are plans as far as I’m aware to deport the testing for the other languages, it just hasn’t been done yet. But if you look at all the patterns on the CDKpatterns.com they all have tests. And there’s multiple different levels of tests you can do. So at the most basic level you can do a snapshot which is to say, “This cloud formation should not change and if it does change my pipeline should break; I don’t want this to deploy.” And for me, that’s the kind of test you put in after your application is stable. You’re not adding any new features but you’re upgrading CDK versions and you just want to make sure nothing breaks. On top of that, you can actually do some more complicated granular tests. So you can go in to write a unit test to say “I expect this cloud formation to have a resource like a Lambda function with this particular handler. And I tend to write a suite of tests all based around that so that that way I can at least know without having to say “this whole cloud formation is the same,” the very important bits on a unit test run I can check that they’re all there. I think that’s awesome.
Jeremy: That’s, again, I mean, just writing tests … that’s one of the things that is so tough with you know the DSLs if you just have cloud formation running the tests on just cloud formation is not very easy to do. So, if you have constructs that are generating then you can run that, that’s a very good approach to doing that.
Matt: We’ve all been there with cloud formation where, as I said, there was a team of four or five of us and somebody would change something that they thought was minor and all of a sudden the cloud formation doesn’t deploy and then we’re all sitting around going “what changed?” So, yeah, that’s why I love the unit test because you know the unit tests are on the cloud formation itself so they’re not on the Java code or the TypeScript code, they’re on the cloud formation. So you know if they pass at least you’ve a valid cloud formation template that should theoretically deploy which is nine tenths of the battle.
Jeremy: Right, right, absolutely. All right so let’s move on to the CDK pattern site. First of all, let’s take a step back. You explained it: it’s a collection of patterns and constructs for CDK that basically outline a number of things. What are there, twenty-three patterns now?
Matt: Twenty-three today, yeah.
Jeremy: Twenty-three patterns. So all these really great patterns, everything from webhooks to cloud formation to Alexa skills. All these other things. So, we kind of got the background of why you build it but really what was really the biggest motivation, what was the trigger point that said I’m going to go out there and spend all of my free time here building this amazing site that people can use.
Matt: Yeah, so I got all this CDK, let’s say it was probably last July or August, was whenever I really got into CDK and at that point I did not think I was going to start going to open source for coding. But it was about the start of November I launched that API gateway pattern internally. I was so confident. I was like, “this is it, we’re done—we’ve API gateway and Lambda, we are good.” But I got to go to re:Invent last year, I was so lucky to get one of the tickets. And I realized that most of the problem was actually just the sheer amount of options for developers and what they can do. You know, it doesn’t matter who you follow, if it’s yourself or any of the advocates or who, there’s a million and one different patterns and opinions for how you can build these things so I was sitting there and I thought how can I help this situation, because as I mentioned earlier, you get a pattern, say I took the simple web service from yourself which is just API gateway Lambda DynomoDb but if I try to deploy that internally I personally know all of the extra steps I have to go to deploy that at Liberty Mutual, but how could I help all the other engineers know that.
So that’s why I had this moment of decision where I could either do it internally or I could go externally and go open source. And there’s a tweet out there from I think January where I said, “Sod it. Be the change you want to see in the world.” I bought CDKpatterns.com and let’s get this created. And ever since then, as you say, I’ve had no free time, pretty much every free second just reading and coding. But the reason why I’ve kept doing it is because the first couple of patterns launched and I said I was doing it because I wanted the other engineers to know what it is, how they can deploy things, and what they need to configure and it really internally did hit the mark and massively helped. So never mind the fact that externally people have picked up on it and it has made my life so much easier as an architect being able to use it as the base conversation, even the likes of this where we know we have the same language to have these conversations.
Jeremy: Yeah, that’s amazing. So let’s get into CDK Patterns itself. So, first of all, CDKpatterns.com. Go and check it out. It’s not just a list of patterns, it is organized around the well-architected framework as well, the serverless lens of that. So just explain the organization so people get that.
Matt: Yup. So there’s a few different ways you can find patterns on the site. Originally it was just a GitHub repo, and I’ve sort of been agiling my way to a product here. So the first way is you can just view all the patterns. You can just go in and click “view all” and you can just see all the pictures and scroll through them. Or you can go in and view them through the serverless component used. So you could say I need a pattern for CloudFront , go ahead and pick CloudFront and it will filter them by the patterns that use CloudFront. But the most interesting way I’ve been trying to do it is, I talked to Heitor Lessa and said, “I want to try to introduce well-architected to CDK patterns” and originally I had this really like hit thing, but in talking to him it became a case of saying, if I’m conducting a well-architected review and somebody goes through that and they say question two is your problem, how do they know the answer to that question is the particular pattern. So that’s why if you go to the well-architected section of CDKpatterns.com it’s broken down by each question in the well-architected serverless lens and then each question doesn’t have one answer it has multiple best practices. So underneath each question there’s a best practice and if I have a pattern that helps with that best practice it’s right there, if I don’t it links to all the AWS docs so you should at least be able to find something.
Jeremy: Awesome. Let’s talk about some of these patterns. So like we said, there are twenty-three of them right now. So what are some of your favorite ones?
Matt: Yeah. The one that took the most effort, which I don’t know if people realize looking at it, was the CloudWatch dashboard pattern.
Jeremy: Right. This is a good pattern.
Matt: That took me probably the longest of all the patterns, I’d say three and a half weeks of just reading theory. So it’s not so much the implementation but what to implement was the problem there, and that goes back to it’s taking the simple web service that I mentioned earlier, API gateway Lambda DynomoDb but if you want to build a CloudWatch dashboard for that what are the right graphs, what are the right metrics, what are the right alerts that you’d want the test to tell you. So I had to get into metric max and you know, try to work out what do I want to alert on and how frequently. So we’ve used that pattern now internally and I personally it’s one of my favorites because it seems like it’s low-hanging fruit but there’s actually a lot more behind it once you scratch the surface.
Jeremy: Right. And I mean, there was just a tall the other day at Serverless Days Virtual from the team at Lego and one of their audit processes was figuring out the observability piece and what metrics they wanted to track and what they wanted to alert on and make sure that everything was with that. That’s a really interesting benefit of the CDK is to say, “Look, here’s the pattern that I’m launching, the connection of components that I’m launching,” but then everything else that’s around that. Just the CloudWatch metrics alone are huge. So, again, super-cool pattern for that particular implementation but a really good framework to use if you’re building out your own constructs for your own company in terms of what other patterns you might need there or what other alerts and metrics surrounding that.
Matt: Yeah, I mean, if you jump into the pattern you’ll see the list of external references are quite long in that pattern and that’s just because there’s a lot of opinions about what the right metrics are but I will say you are going to have to write your own metrics just because the ones out of the box they’re not really good enough today to just use the pure metrics. That’s why you have to get into metric max and fully understand why you’re alerting on what it is. So the pattern breaks it down the metrics for the gateway itself, the metrics form the Lambda function and the metrics for DynamoDb. So if you’ve any of those three it’s worth checking it out and seeing if there’s anything that can help you.
Jeremy: Right. Yeah, and you doing all that research for us is very, very helpful. All right, so another pattern on there that’s one of my favorites, and this is newer to the CDK patterns site, is the Lambda circuit breaker and you based this pattern off of something Gunner Grosch just released. He released a little script, a little npn package, to allow you to implement Lambda circuit breakers, which I’ve been talking about for, I don’t know, two and half years now or something like that. So, I love that this now is encapsulated in the CDK.
Matt: Yeah, it’s awesome. Whenever Gunner released that I sent him a DM, I was just “Okay, do you mind if I throw the CDK pattern for this,” because internally, you have no idea how many conversations I’ve had internally, about needing circuit breakers. I mean, it should be core at the point but it’s not so that’s why whenever I saw that library I thought, you know what, this is something that it’s not a huge effort for me to put out there but it’s of huge value to the community to be able to say if you’re going to integrate with something that may or may not be reliable, let’s give you a mechanism to decide what to do whenever it’s not at its reliable phase. It seems basic but it’s not.
Jeremy: Right. Yeah, and I still don’t understand why this is not built in to AWS or any of these cloud providers because if people are not familiar with the circuit breakers essentially what it is is when you are reaching out to third-party APIs or anything that, again, you could overwhelm, or could go down, you’re building in a mechanism here that says once I start getting a certain amount of failures I’m either going to back off or I’m going to take a break for a minute and then I will keep trying to see if it’s working but I’ll be that good netizen, if you want to use that word, and not overwhelm these downstream services because the worst this you can do is for a service that is responding slowly is send more traffic to it. So, yes, these are the things that it’s such a common thing. You’re always calling third-party APIs, you’re either writing data, you’re reading data. It will be interesting to see what they do with EventBridge and SAS integration and some of those things around that, but until then, Lambda circuit breakers, check them out, awesome pattern.
All right, what's another pattern that you like?
Matt: Another one I like, they’re all based around well-architected, but the Saga Step function. The Saga pattern, this has been around a lot longer than serverless, this is just a design pattern where for … it’s about managing these big long distributed processes of being able to say what happens when there’s failure when you’re midway through the process. For the example in this pattern, it’s a holiday booking, it’s a case of booking your flights and your hotel and I think your car maybe, I can’t remember if they added the car or not, or if it was one layer too many. But the idea is for every step that you take you need to have an equal but opposite undo step, so that if anything goes wrong it unravels layer by layer, so the system is in the state it was before you tried anything. It links to an awesome talk from GOTO by Caitie McCaffrey. And the Saga pattern itself it’s … if you want to build systems and you’re working across these multiple bounded contexts as opposed to one single unit, it’s something you really need to consider.
Jeremy: Yeah. Absolutely. And the Saga pattern is something, too, from a serverless standpoint, I know that Yan Cui, the Burning Monk, had put some stuff about that in the past, which is just … it’s just a really, really great pattern. If you’re coordinating multiple things, I mean. I’m a big fan of choreography, like using EventBridge and having other systems react. And in most cases that’s fine if someone is going to get a mass mailer or some sort of, maybe a coupon is going to go out if someone makes a certain number of purchases, if that fails it’s probably not the end of the world if that didn’t work or updated in your Salesforce or Marketo. But if the inventory system wasn’t updated, well then that’s a problem. We need to make sure those things are taken care of, so absolutely great patterns.
All right, one more pattern I want to call out. In the beginning of 202 I was thinking that voice control was going to be a hugely important thing. With all of these, I’m going to say … I’m going to mute … hold on, I’m going to mute my Alexa so it doesn’t respond to me. But with all of these voice controlled systems and all this voice interactivity, I think the ability for you to now do really complex tasks with your voice is pretty cool. I didn’t see as much this year as I thought I would. I thought there would be more of a progression. But maybe that was because everybody was locked at home and maybe there wasn’t enough opportunity for these things to grow. But I do think it’s a very cool thing and I think Alexa, obviously, is one of the pieces that’s driving this. You just recently released an Alexa skill, so tell us about that.
Matt: Yeah, what’s cool about this pattern is that I didn’t write it, so this was actually a community contribution from another member of Liberty Mutual’s staff, an IO community builder, Chris Plankey, but what’s awesome about this is we also have two Heroes at Liberty Mutual, Jillian Armstrong and Jillian McCann, both experts in machine learning, lex, and voice. So, since I’ve started CDK Patterns thinking how can I get voice e.i. into these patterns because it is something, it’s going to come at some point and be mainstream. The problem with lex in particular it’s not exactly in cloud formation today. You have to deploy your own custom resource to get it out there. But Alexa was apparently slightly easier, so that’s where Chris Plankey was able to take a very basic skill and you still have to navigate the fact that there is Amazon and there is Amazon Web Services so you need an Amazon developer to deploy it, but outside of that the barrier to deploying it is get your Amazon account, clone pattern deploy, and you’ve got Alexa skill which what it does is just let you say, “CDK Patterns tell me what patterns you have.” But you can adapt that to be whatever you want in your context.
Jeremy: Right, yeah, and I’ve developed a couple of Alexa skills more as a test, I’ve never put one out there for other people to use, but they are very cool. They do get kind of complicated, and you need to … and this is another thing, CDK Patterns isn’t going to help you understand all of the semantics and how you have to create the different questions and all that kind of stuff and all the slots, but it’s great for getting the basics set up and, like you said, just being able to iterate on that and make some changes.
All right, awesome. So, again, CDKpatterns.com, go check it out. There are twenty-three patterns there that will get you started immediately and probably get you to really like the CDK. So, do that. Speaking of people who really like the CDK, another thing that happened earlier this year which is crazy how fast this all happened was the CDK Day. So tell us about the CDK Day.
Matt: Yeah, so this was one day I just had this idea and I thought, “What if we just had one day, we just took the whole day, and just talked about everything CDK.” So I sent a message to a few of my friends and from there it just snowballed. I sort of thought everyone would say no, this is too much effort, but everybody was so supportive. And it was two weeks or four weeks after that message we actually put out the tweet to say CDK Day was happening and then it was twelve weeks after that that the day happened and in that time we had to build the website, get the CFP out there, pick the talks, find a host, get the screening platform up and running, do all the social media. But the community absolutely came through, I mean the numbers of people that signed up was incredible. At the very start for the keynote there was over a thousand people watching for a conference that was brand new and all of the speakers all nailed their particular talks. I couldn’t have asked for more considering it was just a random idea I had one day that snowballed into a large thing.
Jeremy: That’s amazing. And the videos of the talks are up on the CDKday.com, right, so you can go back and watch the replays?
Matt: Yeah, they’re all up on YouTube and you can find them. There’s a rewind page on CDKday.com where they’re all individually up there because something I’ve learned since I was a 5R, back-to-back-to-back of talks. To make it all easier for people to consume, I’ve sliced them all up and reuploaded them so yeah, you can find them all there broken down by speaker name and title.
Jeremy: Awesome. So what are some of the best talks that people should go check out?
Matt: One that is really awesome that’s still in beta or def preview is “CDK Pipelines.” This is the idea that with CDK not only can you use your language to deploy your infrastructure but you can use it for multi-region, multi-account set-ups, so everything in the same code. So Thorsten Hoeger, who is a Hero, he talks through the actual live demo of how to do that with CDK pipeline. It’s a really good talk if you haven’t seen CDK pipelines before, you’ll learn a lot.
There’s a couple of other good ones as well. So, Nader Dabit actually gives a talk on AppSync with CDK. I’m a big fan of trying to get Amplify and CDK to be friends. I’m trying to get this rapid development closer together, but he does a live demo of AppSync which is really good. And then the last one was Elad, who came up with the idea for CDK. He showed off his new project which is called projen. This is like an opinionated way of creating new projects. So you can just go in be like, projen, use CDK module, and it’s like … it abstracts certain things like all of the different dependency versions I told you, it abstracts them into a yaml for you, and you don’t have to manage that anymore. You can just give it your GitHub token and it will keep all your projects up to date and stuff. So it’s a really cool concept.
Jeremy: Awesome. All right. So listen, CDKday.com, CDKpatterens.com. So, Matt, thank you so much for, one, joining me here today, but also for literally the tons of work you have done. I don’t think you quite realize how much good you have done, not just within Liberty Mutual but in the community itself. I mean, there are … the CDK Day, everything you have been doing is amazing. Keep it up and … yeah, just awesome. If people want to find out more about you and all these other projects you’re working on and maybe what you’re taking so that you don’t need to sleep so you can keep working on all of these things, how do they do that?
Matt: Yeah, thank you for that. If you want to find me I’m @nideveloper on Twitter. To be fair, if you type in nideveloper into Google, you’ll find a ton of resources that I’m at. But if you want to keep up to date on the patterns, there’s a CDK patterns Twitter handle. If you want to stay up to date on CDK Day, in case we throw another one, there’s a CDK Day Twitter handle that you can follow. If you want blog posts, you can go to dev.2/nideveloper. That has your bases covered. If you can find me in those places, you can find the others.
Jeremy: Awesome. Well, I will put all of this stuff in the show notes, LinkedIn, the blog, your Twitter, CDK Patterns, CDK Day. Matt, thanks again, really appreciate it.
Matt: Thank you for having me.
This episode is sponsored by TriggerMesh and Amazon Web Services.