The Docker Bake Build tool just went general availability, and I'm excited
about what this means for creating reproducible builds and automation
that can run anywhere CI locally.
I love it.
Really, I'm gonna break down some of the features, the benefits
and walk through some examples.
For over six years, Docker has had this little known tool that we could execute
with a Docker Buildex Bake command.
It's meant to provide even more customization and flexibility than
what we're building with today.
But if Bake has been around all these years, why isn't everyone using it?
To answer that, I'll need to take you back in time to 2018.
Docker was on a streak of releases and it seemed like they were creating new
tools every few months, experimenting with various container tools on top of
Docker engine to see what would stick.
We had a release of the BuildKit project and then Bake came out with
the release of Build X Command, even us Docker captains were having a hard
time keeping up with all the ideas coming out of Docker, Inc. But we also
started to see Docker deprecate products causing some of their less popular tools
to silently stop receiving updates.
So I and others were hesitant to use Bake in any significant way since we
weren't really confident in its future.
And we were kind of right at the time, everyone was talking about
orchestration and Docker didn't mention Bake again in a significant way.
So we just assumed that it would quietly become a deprecated
idea like so many others.
Fast forward to today, and Docker recently announced general
availability of Docker Bake.
They did this as a signal to us in the community that after years of development
and heavy use internally, they're ready to fully support Bake in the community.
And in fact, they're so committed to this that they have announced that
they have a plan to change Compose, to use, bake in the background for
all of its bills in a future release.
Now that I know I can trust in a future for Bake, why do I care?
Like what can it do for me today?
Well, at its Core Bake is a simple command that gets its instructions
from a file formatted in JSON, Compose, or my preference, HCL?
Yes, that's the file format we use with Terraform.
Bake Acts as a kind of BuildKit native make file.
It provides a more structured way to configure builds.
It still uses Docker files as the detailed instructions, and you'll probably have
more stages in your Docker files if you go fancy with it, but it replaces long
complex build commands and build scripts with a more advanced and appropriate tool.
Bake is a superset of Docker build and composed builds, but it
makes those things even easier.
Here's a partial list of things that bake can do for you.
before I get to the list of benefits and examples of how you can use Bake,
I wanna thank today's sponsor me.
I've been making Docker, Kubernetes, and DevOps content since at least
2017, and I welcome you to my community of cloud native DevOps engineers.
Besides this podcast, you can join my 20,000 user Discord server.
Subscribe to my YouTube channel where I'm posting content or
live streams You can also sign up for my low traffic newsletter.
Or buy one of my courses with a coupon.
If you'd like to buy me a coffee and support my work, you can become
a member on YouTube, or on Patreon.
And you'll get exclusive benefits like videos on YouTube or
separate chat rooms on Discord.
I should also mention my High Fives DevOps Guild, which is a monthly
virtual meetup on discord with other DevOps professionals Where we talk
trends, what we're working on, and anything we're really excited about.
I don't talk about that enough, but I've been doing it for years and I always
have fun hosting that group every month.
So if you'd like to join that video call, look for the high fives in the memberships
plans on YouTube, Patreon, or my website.
It's about six bucks a month.
Okay.
Let's talk about bakes benefits and some examples on the YouTube version of this,
which you can see in the show notes with everything else and all the details.
That is something where I go through, I think it's four examples
in detail going through the files.
How it works with Compose.
The difference between the HCL format that I prefer for using Bake,
which is the Terraform language and how that compares to Compose.
Pretty much you can do 90% of this stuff, or maybe 80 to 90% of all this
stuff Bake can do in a Compose file, but I think I much prefer the HCL format.
Because it gives you a separate file that's just focused on building images,
so it doesn't muddy the waters of where the composed file is sort of
service centric and it has the volumes and the services, and it's usually meant
for local dev and that sort of thing.
I like having this Docker dash bake do HCL file in the
same route, director, my repo, where the Dockerfile is and the Docker
ignore, and possibly a composed file.
It just feels right to have that separate file sitting beside the Dockerfile.
But here's a short list of some examples of things you can do with Bake.
A lot of these you can actually do in BuildKit with just Docker build if
you wanna make a really long command.
And you can do a lot of it in Compose if you wanna make the build section of
each composed service really long too.
Alright.
First.
Improve build speed by letting BuildKit automate the caches and
parallelism essentially, because you can give Bake a single command, it
will do potentially multiple builds.
You can have one command building dozens of different images if you
set it up that way, it optimizes the caching and parallelism across
all those different activities.
So that it only has to transfer your local source code once into the BuildKit
engine, and it will parallelize all the different parts of the build,
the different architectures in a way that you can't really get in
advance cases with the Docker build command or with composed builds.
Next you can import or export files outside of the Docker context for
building images, which is a little, I mean, honestly, I would feel like
it's a little bit of an anti-pattern.
'cause I always think of everything in a Docker build is in that directory
or subdirectory that you build from.
That's, that's the context.
But there is an option where you can add multiple context locations in the
file system that are outside that, and when you finish the build, you
can choose to export everything in the image to the host file system.
Maybe you were just building a go or rust or see binary and it's a
single file and you want to put it on GitHub releases for download, right?
So you can kind of do all this at once, uh, except for maybe the
part where you actually upload it to the releases in GitHub.
But you can have it not just in an image.
You can have the results put out to the file system.
Next, there are more options for adding variables to your builds and including
those in functions, which you can even use Golang simplistic functions inside the
HCL format to have advanced programmatic ish flow of how the image gets built.
next, like other ways you can build images in BuildKit.
You can add SBOs and Providence attestations to the images with
little options you can set in the HCL.
Next.
You can tag the image with multiple names, which you could always do, but a
lot of people don't realize you can do that in a single Docker build command,
but you can make a list of all the different names you want to name it.
Maybe you want it to be colon latest, but you also want it to be colon aversion or
a date timestamp, or you want to indicate that it's approved for release in the tag.
Or maybe you have multiple registries, which I see a lot, and you maybe
want to push these certain images.
For maybe an official release, you wanna push 'em up to GitHub Container
Registry, also up to Docker Hub.
And if you're doing something private in your company, you might
wanna push it to an ECR instance as well as for developers and GHCR.
I mean, there's just all sorts of scenarios, but you can do it all at
once essentially, and have a, a full list with variables and a little bit
of advanced functions where you can put in day timestamps and other methods to.
Inject dynamic tags into that file.
Next You can build for multiple architectures concurrently, of course.
So you're always wanting to build these days on arm and a MD. Next, just like you
can use Compose override files if you've ever done that, I'm a big fan of that.
For Compose, for custom local individual developer settings, you just create
a file called Docker Compose override YAML, and then Compose will automatically
pick that up on your local system.
My technique there is I usually put that particular.
File name in the dot get ignore so it never gets back into the repo.
And then I create sort of a sample override file for developers, and
then they will all create their own override that they'll use for
development environment variables.
Well bake has that too.
Next you can set an option to automatically push to registries after
the builds are finished, and that's a part of something called the output
where you can have it, choose to load the image into the Docker image cache,
or export to the file system, or automatically push the registries or do
nothing and just have it leave itself in the BuildKit cash, which I see used.
For building stages of images that maybe aren't meant to go somewhere.
Maybe you have a stage that does a CVE scan or a linter or some other
sort of testing, and you're really just using, at this point, the bake
file and the Dockerfile as a way to automate that stuff in the same
tooling during your build process.
But you don't necessarily need those particular images pushed
anywhere or put anywhere.
You just need to run.
That build stage.
So just leave it in the build cache.
Right?
And without examples, this won't maybe make a lot of sense, but inside the bake
file you have something called targets, which is a particular build and that
build could technically build multiple images or multiple architectures of
images, but it's the one thing that you execute from the bake command.
You can also group multiple targets into groups and build them all at once.
This is really handy for mono repo.
So if you decide that a certain file in the mono repo changes, you wanna build
all images in that mono repo, you can technically do that with one bake command
and just have multiple targets, and then group those into a group and call
that group and it will automatically cash and parallelize all that.
Next.
You can also build images remotely by just pointing bake to a GitHub URL and
then it will build what it finds in there.
And you can do all of that in a single bake command, which you can then throw
into your CI, you can run it locally.
And that's one of the wonderful things of Bake is it is meant to run
everywhere In the true style of Docker, it works on more than your machine.
And you know, all of our different cis nowadays have abstracted the Docker and
BuildKit stuff into YAML or whatever their format is, which is usually YAML nowadays.
And so you're usually doing YAML work in their specific CI format.
And the power of this bake is that you can take a lot of, at least
the build complexity, possibly even more than the build complexity.
since you can make all these fancy Dockerfile stages to do other things,
I. You could choose to use the bake file and then in your CI, let's talk
about GitHub Actions for a second.
You really are just running a Docker buildex bake command, and
you can easily switch cis or run it locally in exactly the same way.
We don't today have that option in GitHub Actions because as far as I
know, act is the only real way to try to run GitHub Actions locally, but it's.
A non-official third party tool that has limits and functionality problems.
So I would absolutely prefer the bake file format there.
And of course, Docker has their own official GitHub Actions, of which one
of them all along has been the bake GitHub action that I haven't been
using because I was for reasons not sure what its purpose or future was.
So I was always using and teaching and showing off examples of using
the Docker build GitHub action.
Which I probably wouldn't be surprised to find out that behind it is really
bake, just running in the background.
But there's now an official bake action, so I'm going to personally be switching
a lot of my stuff over from the Docker build action as I start writing some
of these HCL files for Docker Bake.
I'm gonna switch over to that GitHub action to hopefully reduce the amount.
Of YAML, I have to type in order to get things at GitHub to work.
And then with the advantage of being able to run that exact same build locally.
And like I said earlier, you can watch the YouTube video in in LinkedIn, the
show notes to see some of the examples as well as a little bit of the video of
me showing how I baked cookies, which was just a fun part of that video to make.
All right.
One of my final thoughts about Bake is that to take full advantage of
it as a CI workflow engine, you need to lean into Dockerfile stages.
If you like the idea of a Docker Buildex Bake command being one of the only
things you need to run in your CI.
Then you need to lean into expanding your Dockerfile with testing stages,
maybe CVE scanning stages, et cetera.
Now.
I've given demos at conferences where I do that exact thing with multi-stage
builds, showing off how multi-stage builds work and the things you can do
with them, and trying to shove every possible CI action into a Dockerfile.
But the reality is, is that Dockerfile stages aren't a
great place for a lot of that.
Stuff to live.
A lot of the CI tools actually expect to run from outside
the image once it's built.
So for bigger teams and projects, it kind of muddies the waters between
building your app image and all the other activities you want to happen In CI, do we
want the single file format of Dockerfile to be that single source of truth for
everything and then you end up with a possible 300 or 400 line Dockerfile.
Also CI solutions don't monitor BuildKit well.
So to them, this just looks like one big pipeline step rather than
the detailed view you would get if each action was its own CI step.
I've noticed that over the years as I've tried to make advanced Docker
builds, and it's unfortunate that the CI companies don't do more to see into
what BuildKit is doing to give you that advanced workflow or pipeline view.
This really comes down to preference of where you like to spend
your time creating automation.
Do you love the Dockerfile format and want to use it to the max?
Great.
Use Bake for as much as you can.
Do you use GitHub Actions for CI and prefer the YAML job
structure of that solution?
Great.
Do that.
I can tell you that if you like the idea of bake, because you can run it
anywhere and you can make it declarative and repeatable for more things than
just image builds, but maybe you're not a fan of doing it all in Docker
files, then you might wanna look at more advanced tools like Dagger that
allow you to run all these things.
With the language your app is written in.
More on that in another video.
Thanks for listening.
I'll see you in the next one.