WEBVTT NOTE This file was generated by Descript 00:00:04.414 --> 00:00:08.764 The Docker Bake Build tool just went general availability, and I'm excited 00:00:08.764 --> 00:00:12.154 about what this means for creating reproducible builds and automation 00:00:12.154 --> 00:00:15.154 that can run anywhere CI locally. 00:00:15.364 --> 00:00:16.054 I love it. 00:00:16.399 --> 00:00:19.709 Really, I'm gonna break down some of the features, the benefits 00:00:19.949 --> 00:00:21.179 and walk through some examples. 00:00:21.924 --> 00:00:25.494 For over six years, Docker has had this little known tool that we could execute 00:00:25.494 --> 00:00:27.804 with a Docker Buildex Bake command. 00:00:28.674 --> 00:00:31.614 It's meant to provide even more customization and flexibility than 00:00:31.614 --> 00:00:32.964 what we're building with today. 00:00:33.454 --> 00:00:37.144 But if Bake has been around all these years, why isn't everyone using it? 00:00:37.414 --> 00:00:41.284 To answer that, I'll need to take you back in time to 2018. 00:00:44.841 --> 00:00:47.811 Docker was on a streak of releases and it seemed like they were creating new 00:00:47.811 --> 00:00:51.141 tools every few months, experimenting with various container tools on top of 00:00:51.141 --> 00:00:52.551 Docker engine to see what would stick. 00:00:52.941 --> 00:00:56.511 We had a release of the BuildKit project and then Bake came out with 00:00:56.511 --> 00:01:00.751 the release of Build X Command, even us Docker captains were having a hard 00:01:00.751 --> 00:01:04.271 time keeping up with all the ideas coming out of Docker, Inc. But we also 00:01:04.271 --> 00:01:08.142 started to see Docker deprecate products causing some of their less popular tools 00:01:08.472 --> 00:01:10.242 to silently stop receiving updates. 00:01:10.242 --> 00:01:14.682 So I and others were hesitant to use Bake in any significant way since we 00:01:14.682 --> 00:01:16.212 weren't really confident in its future. 00:01:16.489 --> 00:01:19.609 And we were kind of right at the time, everyone was talking about 00:01:19.609 --> 00:01:23.869 orchestration and Docker didn't mention Bake again in a significant way. 00:01:23.869 --> 00:01:27.019 So we just assumed that it would quietly become a deprecated 00:01:27.049 --> 00:01:28.399 idea like so many others. 00:01:33.626 --> 00:01:36.716 Fast forward to today, and Docker recently announced general 00:01:36.716 --> 00:01:38.396 availability of Docker Bake. 00:01:38.726 --> 00:01:42.416 They did this as a signal to us in the community that after years of development 00:01:42.566 --> 00:01:46.599 and heavy use internally, they're ready to fully support Bake in the community. 00:01:46.599 --> 00:01:49.869 And in fact, they're so committed to this that they have announced that 00:01:49.869 --> 00:01:54.009 they have a plan to change Compose, to use, bake in the background for 00:01:54.009 --> 00:01:55.839 all of its bills in a future release. 00:01:56.232 --> 00:02:00.462 Now that I know I can trust in a future for Bake, why do I care? 00:02:00.462 --> 00:02:02.412 Like what can it do for me today? 00:02:02.742 --> 00:02:06.582 Well, at its Core Bake is a simple command that gets its instructions 00:02:06.582 --> 00:02:11.222 from a file formatted in JSON, Compose, or my preference, HCL? 00:02:11.342 --> 00:02:13.257 Yes, that's the file format we use with Terraform. 00:02:13.680 --> 00:02:16.920 Bake Acts as a kind of BuildKit native make file. 00:02:17.130 --> 00:02:19.770 It provides a more structured way to configure builds. 00:02:20.010 --> 00:02:24.960 It still uses Docker files as the detailed instructions, and you'll probably have 00:02:24.960 --> 00:02:29.249 more stages in your Docker files if you go fancy with it, but it replaces long 00:02:29.249 --> 00:02:33.539 complex build commands and build scripts with a more advanced and appropriate tool. 00:02:33.989 --> 00:02:37.919 Bake is a superset of Docker build and composed builds, but it 00:02:37.919 --> 00:02:39.779 makes those things even easier. 00:02:40.304 --> 00:02:43.694 Here's a partial list of things that bake can do for you. 00:02:44.481 --> 00:02:47.811 before I get to the list of benefits and examples of how you can use Bake, 00:02:48.201 --> 00:02:50.841 I wanna thank today's sponsor me. 00:02:51.291 --> 00:02:55.281 I've been making Docker, Kubernetes, and DevOps content since at least 00:02:55.281 --> 00:02:59.871 2017, and I welcome you to my community of cloud native DevOps engineers. 00:03:00.201 --> 00:03:03.921 Besides this podcast, you can join my 20,000 user Discord server. 00:03:04.386 --> 00:03:07.056 Subscribe to my YouTube channel where I'm posting content or 00:03:07.056 --> 00:03:10.195 live streams You can also sign up for my low traffic newsletter. 00:03:10.368 --> 00:03:12.168 Or buy one of my courses with a coupon. 00:03:12.340 --> 00:03:15.820 If you'd like to buy me a coffee and support my work, you can become 00:03:15.820 --> 00:03:18.420 a member on YouTube, or on Patreon. 00:03:18.500 --> 00:03:22.700 And you'll get exclusive benefits like videos on YouTube or 00:03:22.700 --> 00:03:24.530 separate chat rooms on Discord. 00:03:24.830 --> 00:03:28.910 I should also mention my High Fives DevOps Guild, which is a monthly 00:03:28.910 --> 00:03:32.870 virtual meetup on discord with other DevOps professionals Where we talk 00:03:32.870 --> 00:03:36.500 trends, what we're working on, and anything we're really excited about. 00:03:36.860 --> 00:03:40.460 I don't talk about that enough, but I've been doing it for years and I always 00:03:40.460 --> 00:03:42.410 have fun hosting that group every month. 00:03:42.650 --> 00:03:46.970 So if you'd like to join that video call, look for the high fives in the memberships 00:03:46.970 --> 00:03:50.846 plans on YouTube, Patreon, or my website. 00:03:51.086 --> 00:03:52.286 It's about six bucks a month. 00:03:53.085 --> 00:03:53.535 Okay. 00:03:53.745 --> 00:03:59.445 Let's talk about bakes benefits and some examples on the YouTube version of this, 00:03:59.445 --> 00:04:02.445 which you can see in the show notes with everything else and all the details. 00:04:02.656 --> 00:04:06.076 That is something where I go through, I think it's four examples 00:04:06.076 --> 00:04:08.086 in detail going through the files. 00:04:08.431 --> 00:04:09.871 How it works with Compose. 00:04:09.871 --> 00:04:15.121 The difference between the HCL format that I prefer for using Bake, 00:04:15.121 --> 00:04:19.441 which is the Terraform language and how that compares to Compose. 00:04:19.771 --> 00:04:24.121 Pretty much you can do 90% of this stuff, or maybe 80 to 90% of all this 00:04:24.121 --> 00:04:28.651 stuff Bake can do in a Compose file, but I think I much prefer the HCL format. 00:04:28.651 --> 00:04:33.451 Because it gives you a separate file that's just focused on building images, 00:04:33.451 --> 00:04:37.047 so it doesn't muddy the waters of where the composed file is sort of 00:04:37.047 --> 00:04:40.837 service centric and it has the volumes and the services, and it's usually meant 00:04:40.837 --> 00:04:42.907 for local dev and that sort of thing. 00:04:42.957 --> 00:04:48.057 I like having this Docker dash bake do HCL file in the 00:04:48.057 --> 00:04:50.787 same route, director, my repo, where the Dockerfile is and the Docker 00:04:50.787 --> 00:04:52.527 ignore, and possibly a composed file. 00:04:52.737 --> 00:04:56.627 It just feels right to have that separate file sitting beside the Dockerfile. 00:04:56.992 --> 00:05:01.042 But here's a short list of some examples of things you can do with Bake. 00:05:01.162 --> 00:05:04.132 A lot of these you can actually do in BuildKit with just Docker build if 00:05:04.132 --> 00:05:05.932 you wanna make a really long command. 00:05:06.079 --> 00:05:09.259 And you can do a lot of it in Compose if you wanna make the build section of 00:05:09.259 --> 00:05:11.719 each composed service really long too. 00:05:12.199 --> 00:05:12.413 Alright. 00:05:12.653 --> 00:05:13.093 First. 00:05:14.062 --> 00:05:18.321 Improve build speed by letting BuildKit automate the caches and 00:05:18.321 --> 00:05:23.327 parallelism essentially, because you can give Bake a single command, it 00:05:23.327 --> 00:05:25.367 will do potentially multiple builds. 00:05:25.367 --> 00:05:28.807 You can have one command building dozens of different images if you 00:05:28.807 --> 00:05:33.277 set it up that way, it optimizes the caching and parallelism across 00:05:33.607 --> 00:05:35.047 all those different activities. 00:05:35.217 --> 00:05:38.877 So that it only has to transfer your local source code once into the BuildKit 00:05:38.877 --> 00:05:42.807 engine, and it will parallelize all the different parts of the build, 00:05:42.807 --> 00:05:46.737 the different architectures in a way that you can't really get in 00:05:46.737 --> 00:05:50.847 advance cases with the Docker build command or with composed builds. 00:05:51.291 --> 00:05:56.001 Next you can import or export files outside of the Docker context for 00:05:56.001 --> 00:05:59.661 building images, which is a little, I mean, honestly, I would feel like 00:05:59.661 --> 00:06:01.071 it's a little bit of an anti-pattern. 00:06:01.071 --> 00:06:04.461 'cause I always think of everything in a Docker build is in that directory 00:06:04.461 --> 00:06:06.141 or subdirectory that you build from. 00:06:06.141 --> 00:06:07.521 That's, that's the context. 00:06:07.911 --> 00:06:12.281 But there is an option where you can add multiple context locations in the 00:06:12.281 --> 00:06:16.722 file system that are outside that, and when you finish the build, you 00:06:16.722 --> 00:06:21.582 can choose to export everything in the image to the host file system. 00:06:21.852 --> 00:06:26.712 Maybe you were just building a go or rust or see binary and it's a 00:06:26.712 --> 00:06:31.602 single file and you want to put it on GitHub releases for download, right? 00:06:31.602 --> 00:06:34.452 So you can kind of do all this at once, uh, except for maybe the 00:06:34.452 --> 00:06:37.032 part where you actually upload it to the releases in GitHub. 00:06:37.272 --> 00:06:39.462 But you can have it not just in an image. 00:06:39.462 --> 00:06:41.652 You can have the results put out to the file system. 00:06:42.388 --> 00:06:46.768 Next, there are more options for adding variables to your builds and including 00:06:46.768 --> 00:06:52.768 those in functions, which you can even use Golang simplistic functions inside the 00:06:52.768 --> 00:06:59.683 HCL format to have advanced programmatic ish flow of how the image gets built. 00:07:00.418 --> 00:07:03.328 next, like other ways you can build images in BuildKit. 00:07:03.358 --> 00:07:08.008 You can add SBOs and Providence attestations to the images with 00:07:08.008 --> 00:07:09.838 little options you can set in the HCL. 00:07:10.700 --> 00:07:11.090 Next. 00:07:11.090 --> 00:07:14.900 You can tag the image with multiple names, which you could always do, but a 00:07:14.900 --> 00:07:17.540 lot of people don't realize you can do that in a single Docker build command, 00:07:18.020 --> 00:07:21.650 but you can make a list of all the different names you want to name it. 00:07:21.650 --> 00:07:26.750 Maybe you want it to be colon latest, but you also want it to be colon aversion or 00:07:26.750 --> 00:07:31.310 a date timestamp, or you want to indicate that it's approved for release in the tag. 00:07:31.520 --> 00:07:35.240 Or maybe you have multiple registries, which I see a lot, and you maybe 00:07:35.240 --> 00:07:37.370 want to push these certain images. 00:07:37.532 --> 00:07:40.682 For maybe an official release, you wanna push 'em up to GitHub Container 00:07:40.682 --> 00:07:42.902 Registry, also up to Docker Hub. 00:07:43.028 --> 00:07:45.668 And if you're doing something private in your company, you might 00:07:45.668 --> 00:07:50.378 wanna push it to an ECR instance as well as for developers and GHCR. 00:07:50.378 --> 00:07:53.228 I mean, there's just all sorts of scenarios, but you can do it all at 00:07:53.228 --> 00:07:58.028 once essentially, and have a, a full list with variables and a little bit 00:07:58.028 --> 00:08:02.228 of advanced functions where you can put in day timestamps and other methods to. 00:08:02.228 --> 00:08:04.418 Inject dynamic tags into that file. 00:08:05.308 --> 00:08:08.848 Next You can build for multiple architectures concurrently, of course. 00:08:08.848 --> 00:08:14.212 So you're always wanting to build these days on arm and a MD. Next, just like you 00:08:14.212 --> 00:08:18.202 can use Compose override files if you've ever done that, I'm a big fan of that. 00:08:18.202 --> 00:08:22.402 For Compose, for custom local individual developer settings, you just create 00:08:22.402 --> 00:08:28.279 a file called Docker Compose override YAML, and then Compose will automatically 00:08:28.279 --> 00:08:29.659 pick that up on your local system. 00:08:29.959 --> 00:08:32.599 My technique there is I usually put that particular. 00:08:32.709 --> 00:08:36.509 File name in the dot get ignore so it never gets back into the repo. 00:08:36.509 --> 00:08:39.749 And then I create sort of a sample override file for developers, and 00:08:39.749 --> 00:08:42.749 then they will all create their own override that they'll use for 00:08:42.749 --> 00:08:44.309 development environment variables. 00:08:44.492 --> 00:08:45.962 Well bake has that too. 00:08:46.837 --> 00:08:50.957 Next you can set an option to automatically push to registries after 00:08:50.957 --> 00:08:54.017 the builds are finished, and that's a part of something called the output 00:08:54.317 --> 00:08:58.740 where you can have it, choose to load the image into the Docker image cache, 00:08:58.797 --> 00:09:03.057 or export to the file system, or automatically push the registries or do 00:09:03.057 --> 00:09:08.667 nothing and just have it leave itself in the BuildKit cash, which I see used. 00:09:08.667 --> 00:09:13.287 For building stages of images that maybe aren't meant to go somewhere. 00:09:13.287 --> 00:09:18.567 Maybe you have a stage that does a CVE scan or a linter or some other 00:09:18.567 --> 00:09:23.007 sort of testing, and you're really just using, at this point, the bake 00:09:23.007 --> 00:09:26.997 file and the Dockerfile as a way to automate that stuff in the same 00:09:26.997 --> 00:09:29.217 tooling during your build process. 00:09:29.487 --> 00:09:32.457 But you don't necessarily need those particular images pushed 00:09:32.457 --> 00:09:33.657 anywhere or put anywhere. 00:09:33.867 --> 00:09:35.097 You just need to run. 00:09:35.097 --> 00:09:36.057 That build stage. 00:09:36.057 --> 00:09:37.437 So just leave it in the build cache. 00:09:37.437 --> 00:09:37.797 Right? 00:09:37.797 --> 00:09:41.577 And without examples, this won't maybe make a lot of sense, but inside the bake 00:09:41.577 --> 00:09:46.337 file you have something called targets, which is a particular build and that 00:09:46.337 --> 00:09:50.177 build could technically build multiple images or multiple architectures of 00:09:50.177 --> 00:09:54.377 images, but it's the one thing that you execute from the bake command. 00:09:54.737 --> 00:09:59.657 You can also group multiple targets into groups and build them all at once. 00:09:59.657 --> 00:10:02.657 This is really handy for mono repo. 00:10:02.657 --> 00:10:07.067 So if you decide that a certain file in the mono repo changes, you wanna build 00:10:07.067 --> 00:10:11.687 all images in that mono repo, you can technically do that with one bake command 00:10:11.927 --> 00:10:16.544 and just have multiple targets, and then group those into a group and call 00:10:16.544 --> 00:10:20.144 that group and it will automatically cash and parallelize all that. 00:10:21.056 --> 00:10:21.356 Next. 00:10:21.356 --> 00:10:27.386 You can also build images remotely by just pointing bake to a GitHub URL and 00:10:27.386 --> 00:10:29.576 then it will build what it finds in there. 00:10:29.576 --> 00:10:34.736 And you can do all of that in a single bake command, which you can then throw 00:10:34.763 --> 00:10:37.043 into your CI, you can run it locally. 00:10:37.043 --> 00:10:39.863 And that's one of the wonderful things of Bake is it is meant to run 00:10:39.863 --> 00:10:45.383 everywhere In the true style of Docker, it works on more than your machine. 00:10:46.049 --> 00:10:50.549 And you know, all of our different cis nowadays have abstracted the Docker and 00:10:50.549 --> 00:10:55.679 BuildKit stuff into YAML or whatever their format is, which is usually YAML nowadays. 00:10:55.679 --> 00:11:00.989 And so you're usually doing YAML work in their specific CI format. 00:11:00.989 --> 00:11:04.649 And the power of this bake is that you can take a lot of, at least 00:11:04.649 --> 00:11:08.189 the build complexity, possibly even more than the build complexity. 00:11:08.239 --> 00:11:11.299 since you can make all these fancy Dockerfile stages to do other things, 00:11:11.484 --> 00:11:15.924 I. You could choose to use the bake file and then in your CI, let's talk 00:11:15.924 --> 00:11:17.244 about GitHub Actions for a second. 00:11:17.664 --> 00:11:21.684 You really are just running a Docker buildex bake command, and 00:11:21.684 --> 00:11:25.464 you can easily switch cis or run it locally in exactly the same way. 00:11:25.794 --> 00:11:29.794 We don't today have that option in GitHub Actions because as far as I 00:11:29.794 --> 00:11:33.994 know, act is the only real way to try to run GitHub Actions locally, but it's. 00:11:33.994 --> 00:11:37.804 A non-official third party tool that has limits and functionality problems. 00:11:37.804 --> 00:11:40.984 So I would absolutely prefer the bake file format there. 00:11:40.984 --> 00:11:44.914 And of course, Docker has their own official GitHub Actions, of which one 00:11:44.914 --> 00:11:49.754 of them all along has been the bake GitHub action that I haven't been 00:11:49.754 --> 00:11:54.334 using because I was for reasons not sure what its purpose or future was. 00:11:54.544 --> 00:11:58.804 So I was always using and teaching and showing off examples of using 00:11:58.804 --> 00:12:01.564 the Docker build GitHub action. 00:12:01.564 --> 00:12:04.604 Which I probably wouldn't be surprised to find out that behind it is really 00:12:04.604 --> 00:12:05.824 bake, just running in the background. 00:12:06.064 --> 00:12:10.534 But there's now an official bake action, so I'm going to personally be switching 00:12:10.534 --> 00:12:15.934 a lot of my stuff over from the Docker build action as I start writing some 00:12:15.934 --> 00:12:18.034 of these HCL files for Docker Bake. 00:12:18.034 --> 00:12:21.634 I'm gonna switch over to that GitHub action to hopefully reduce the amount. 00:12:21.634 --> 00:12:25.204 Of YAML, I have to type in order to get things at GitHub to work. 00:12:25.294 --> 00:12:28.774 And then with the advantage of being able to run that exact same build locally. 00:12:28.774 --> 00:12:32.224 And like I said earlier, you can watch the YouTube video in in LinkedIn, the 00:12:32.224 --> 00:12:36.334 show notes to see some of the examples as well as a little bit of the video of 00:12:36.334 --> 00:12:41.254 me showing how I baked cookies, which was just a fun part of that video to make. 00:12:41.254 --> 00:12:41.644 All right. 00:12:41.884 --> 00:12:45.424 One of my final thoughts about Bake is that to take full advantage of 00:12:45.424 --> 00:12:50.464 it as a CI workflow engine, you need to lean into Dockerfile stages. 00:12:50.644 --> 00:12:55.174 If you like the idea of a Docker Buildex Bake command being one of the only 00:12:55.174 --> 00:12:57.214 things you need to run in your CI. 00:12:58.069 --> 00:13:02.089 Then you need to lean into expanding your Dockerfile with testing stages, 00:13:02.269 --> 00:13:05.209 maybe CVE scanning stages, et cetera. 00:13:05.459 --> 00:13:05.609 Now. 00:13:05.783 --> 00:13:10.253 I've given demos at conferences where I do that exact thing with multi-stage 00:13:10.253 --> 00:13:14.273 builds, showing off how multi-stage builds work and the things you can do 00:13:14.273 --> 00:13:19.013 with them, and trying to shove every possible CI action into a Dockerfile. 00:13:19.043 --> 00:13:22.433 But the reality is, is that Dockerfile stages aren't a 00:13:22.433 --> 00:13:24.143 great place for a lot of that. 00:13:24.168 --> 00:13:24.918 Stuff to live. 00:13:25.248 --> 00:13:28.308 A lot of the CI tools actually expect to run from outside 00:13:28.308 --> 00:13:29.598 the image once it's built. 00:13:30.018 --> 00:13:34.218 So for bigger teams and projects, it kind of muddies the waters between 00:13:34.218 --> 00:13:39.828 building your app image and all the other activities you want to happen In CI, do we 00:13:39.978 --> 00:13:44.808 want the single file format of Dockerfile to be that single source of truth for 00:13:44.808 --> 00:13:48.948 everything and then you end up with a possible 300 or 400 line Dockerfile. 00:13:49.264 --> 00:13:52.684 Also CI solutions don't monitor BuildKit well. 00:13:52.834 --> 00:13:57.304 So to them, this just looks like one big pipeline step rather than 00:13:57.304 --> 00:14:01.744 the detailed view you would get if each action was its own CI step. 00:14:02.194 --> 00:14:05.254 I've noticed that over the years as I've tried to make advanced Docker 00:14:05.254 --> 00:14:10.534 builds, and it's unfortunate that the CI companies don't do more to see into 00:14:10.534 --> 00:14:14.404 what BuildKit is doing to give you that advanced workflow or pipeline view. 00:14:14.963 --> 00:14:17.813 This really comes down to preference of where you like to spend 00:14:17.813 --> 00:14:19.133 your time creating automation. 00:14:19.403 --> 00:14:22.343 Do you love the Dockerfile format and want to use it to the max? 00:14:22.343 --> 00:14:22.733 Great. 00:14:23.003 --> 00:14:24.563 Use Bake for as much as you can. 00:14:25.043 --> 00:14:29.213 Do you use GitHub Actions for CI and prefer the YAML job 00:14:29.213 --> 00:14:30.413 structure of that solution? 00:14:30.713 --> 00:14:31.163 Great. 00:14:31.403 --> 00:14:31.973 Do that. 00:14:32.133 --> 00:14:35.643 I can tell you that if you like the idea of bake, because you can run it 00:14:35.643 --> 00:14:39.843 anywhere and you can make it declarative and repeatable for more things than 00:14:39.843 --> 00:14:44.283 just image builds, but maybe you're not a fan of doing it all in Docker 00:14:44.283 --> 00:14:48.966 files, then you might wanna look at more advanced tools like Dagger that 00:14:48.966 --> 00:14:50.826 allow you to run all these things. 00:14:50.826 --> 00:14:53.436 With the language your app is written in. 00:14:53.886 --> 00:14:55.176 More on that in another video. 00:14:56.283 --> 00:14:57.093 Thanks for listening. 00:14:57.213 --> 00:14:58.013 I'll see you in the next one.