This is DevOps and Docker talk, and I am your host, Bret Fisher.
This was a fun episode this week.
Nirmal Mehta is back from AWS and our guest this week
was Michael Irwin of Docker.
He is a recurring guest if you've been listening to this podcast for any length
of time, I think we had him on earlier this year, and he's been a friend for a
decade, former Docker captain, now Docker employee advocating for Docker everywhere.
He's all over the place and, I always have him on because he
breaks stuff down to help us all understand what's going on at Docker.
And there is a giant list of things we had to talk about this week, all AI related,
all separate pieces of the puzzle that can be used independently or together.
So in this episode, we will be covering the Docker Model Runner, which I've
talked about a lot on this show over the last four months, for running open
or free models locally or remotely on servers or on your machine, wherever.
Basically you get to run your own models and Docker will host them for
you, Docker will encapsulate and allow you to use the Docker CLI to manage it.
Then we have the hub model catalogs, so you can pick one of the dozens of
models I guess, that are available in Docker Hub and we talk about Gordon
ai, which is their chat bot built into Docker Desktop and the Docker CLI.
We then get into MCP toolkit and the hubs MCP catalog, and how to bring all
our tools into our local ai or like, using some other AI and you're just
wanting to use your own MCP tools.
We talk about how Docker manages all that and it uses the MCP gateway
that they recently open sourced to front end all those tools and
to help you manage your MCP tools.
We also then get into compose and how you add models and the MCP gateway plus
MCP tools, all into your composed files.
Then we talk about how to use that for building agents a little bit
And then Offload, which allows you to build, run containers,
run Docker models, all in Docker Cloud, and they call it Offload
It actually is a pretty unique name.
Most other people just called it Docker Cloud, but they call it Docker Offload.
Great name.
You just toggle inside your Docker UI and you're good to go.
Everything's remote.
And then we at the very end talked about Compose Now works at least
the YAML files of Compose, now work inside of Google Cloud Run.
Probably coming to other places soon.
So we go for almost 90 minutes on this show because for good reason, we
talk about the use cases of all these different various AI parts of the puzzle.
And I'm excited because it paints, I hope for you a complete picture
of what Docker has released, how it all works together, when you would
choose each part for problem solving, different problems and all that.
So please enjoy this episode with Nirmal and Michael Irwin of Docker.
Hi!
Hey, Awesome.
I'm doing all right.
we have a special guest Michael, another Virginian.
Hello.
Hello,
we all have known each other a decade.
So Michael, tell us who you are
so I'm, Michael Irwin, I work at Docker.
I'm in our developer success team, and do a lot of teaching, training,
education, breaking down the stuff, trying to make it somewhat understandable.
I've been with Docker for almost three and a half years now, so it's
gone pretty quickly, but, it's fun to spend time with the community
and just help developers out.
Learn about this stuff, but also how do you actually take advantage
and do cool stuff with it?
That's awesome.
I mean, this is like a new wave of, essentially development
tooling and, it's pretty exciting.
Yeah.
we've got a list people.
I only make lists on this show a couple of times a year.
Yeah, it does seem like every time I'm on the show, there's a list involved.
yeah, that's a pretty decent list.
let's break it down because we got a lot to get through and we realize
that on Docker's channel and on this channel, They've been talking
about all these fun features.
I've talked at length about Docker Model Runner.
Hub Catalog, I don't have a bunch of videos on MCP or the MCP tools, but
actually we've had streams on it before.
But, I think the MCP toolkit and the MCP gateway I'm most excited about that.
So we're going to get to that in a little bit.
I think, technically, the first thing out of the gate was Gordon AI.
this to me is.
From a user's perspective, it's a Docker focused or maybe a developer focused
AI chatbot, essentially similar to ChatGPT, but it's in my Docker interface.
if I'm staring at the Docker desktop interface, there's an ask Gordon button.
And if you've never clicked that, or if you clicked it once a couple
years ago, it has changed a lot.
There have been enhancements so o now we have memory and threads and it saves
me from having to go to ChatGPT when I'm working, specifically around Docker
stuff, but anything it feels like I can ask it developer related stuff
Can you tell me now, is this free for everyone?
How does this play out in terms of who can use this
Yeah, so everybody can access it right now.
The only limitations may be some of our business orgs.
those orgs have to, enable it.
We're just not going to roll out all the AI stuff as most organizations
are pretty cautious about that.
but yeah, Gordon's available to everybody.
it started off actually mostly as a documentation help of just help me
answer questions and keep up with new features and that kind of stuff.
But, as you've noted, we've added new capabilities and new tools along the way.
I was doing some experiments just the other day of Hey Gordon, help
me convert this Dockerfile to use the new Docker hardened images.
And it would do the conversion, find the images that my organization has that match
the one that I'm using this Dockerfile.
you're starting to see more and more capabilities built into it as well it's
a pretty fun little assistant there.
Yeah,
highly recommend folks that are listening that if you've not touched Docker and
you just open it up, check out Gordon and ask those questions that you probably
have, compose all the things you could probably put that in there and Gordon will
probably try to compose all the things.
you know what one of my common uses for this is I need to translate a docker
run command into a compose file or back and forth like, please give me the
docker run equivalent of this compose file or please turn these docker run
commands or this docker build command into a compose file and it saves me.
It's not, I mean, I've been doing this stuff almost every day for a decade,
so it's not like I needed to do it for me, but it's still faster than a pro
typing it in from memory, I'm at the point now where I rarely need to refer
to the docs, but it's still faster than me at writing a Compose file
out and saving it to the hard drive.
You know, because sometimes we debate around whether AI tools are useful for
senior versus junior and blah, blah, blah, and I don't know, as a senior, it saves
me keystrokes, it saves me time, unlike when it first came out a couple years
ago, I have Tracked it, given feedback to the team and it no longer makes Compose
files with old Compose information.
Like it, at least in the
the version tag.
Yeah.
Cause it, the version tag.
Yeah.
I was a big proponent of Hey, this hasn't been in there for five years.
Why is it recommending it?
and they fixed that.
So, I'm very appreciative of it.
We're going to save it, we're going to talk about MCP tools, and then
we're going to come back to this, because it gets, it's gotten better,
and one of the reasons it's gotten better is it can talk to tools.
So I don't want to really spoil that, But yeah, I love the suggestions because
sometimes when you're staring at a blank chat box, it feels kind of like being a
writer and staring at a blank document.
It's like, I know this is supposed to be a cool tool, but
I don't know what to do with it.
do I start?
Yeah.
analogy
there's even some cool spots, like if you go to the containers tab, You'll
see on the right side a little like magic icon there, and then you can
ask questions about that container.
So if you have a container that's failing to start or whatever, you can
basically start a thread around the context of why is this thing not starting.
Interesting.
I have not tried that.
it actually reminds me of the little star we get for Gemini that's in every
single Google app that I'm opening.
I guess we're all, slowly converging on the starlight AI icons now, I
would have thought it would have been a robot face, but we chose this
vague collection of stars and pluses.
I and that is pretty cool.
I, so is there more, is it like an images as well?
So yeah, anywhere you see that icon, you can start a
conversation around that thing.
I'm going to start clicking on that more often to see what it tells me.
Like, Hey, what's the, what data.
Ooh, How do I back up this volume?
That's, that is pretty cool.
So they're
that is the number one question
I love that they're context aware.
Like they know that this is a volume, or at least the prompts.
it's like prompt suggestions, which This feels a little meta.
Is the AI suggesting prompts for the ai?
It's not in this case, but it certainly could.
All right.
so that's ask Gordon,
Woo!
Alright.
And that is what you said.
that is available to everyone running Docker Desktop.
Is that, to be clear, that's not available for people running
Docker Engine on Linux, right?
That is not There is a CLI to it, isn't there?
There's Docker AI.
but that's part of the Docker Desktop installation.
that's a CLI plugin and it's going to talk to the components that are
bundled in Docker Desktop there.
Yeah.
so we have our, this is an AI that's outsourced to Docker.
It's not running locally.
it's just calling APIs from the Docker desktop interface.
So then if I don't want to use Gordon AI, but I want to run my own AI models
locally, which I feel like this is pretty niche because a lot of people I
talk to, their GPU is underwhelming, and they don't have an M4 Mac or a NVIDIA
desktop tower with a giant GPU in it.
granted, I guess there are models that run on CPUs, but I
am not the model expert here.
I've been trying to catch up this year because I actually do have a decent
laptop now with an M4 in it so I can run at least some of the Apple models.
But this Docker model runner.
it's a pretty cool feature, but you can pick your models, so
you can pick a really small one.
On Windows, it has to be an NVIDIA GPU on Windows to work.
Is that right?
So it supports both NVIDIA and Adreno, so if you've got a
Qualcomm chip, it'll work there.
Okay, and on Mac, it just uses system memory because that's how Macs work.
They have the universal memory, right?
And I mean, for me, it's been great because I have a brand
new machine with 48 gig of RAM.
I know that is not normal.
but these models, Mac, it can be a little problematic because it's like, if I have
a bunch of browser tabs open, I can't run the full size model that I want to
run, because that's one of the problems is like, it's all using the same thing.
So I have to do like we did back with VMs, I have to shut all my apps down
and then run, because I always want to run the biggest model possible.
So I have this like 40, 46 gig model.
I think it's the Devstral new model that's supposed to be
great for local development.
So for those of you listening, if you want to go in depth, we're not going to
have time for that, but it is, Docker Model Runner lets you run models locally.
you can technically run this now on Docker infrastructure, right?
Which we're going to get to in a little bit.
dun dun.
Okay.
yeah, there is this thing called offload.
Michael's going to know way more about that.
Cause that's a really new feature, but you don't have to actually
run these models locally now.
Like you could offload them.
Right.
Okay.
What are you seeing?
are people coming to Docker, like trying to run the biggest models possible?
They got like multiple GPUs or are you just seeing people kind of tinkering out?
What's some analogies there?
there's a little bit of everything.
I'd say a lot of folks are still exploring the space and figure out what's the
right ways of doing things as well too.
So, one of the interesting things is, and one of the things to keep in mind
is that Let's break outta the AI space.
when you say, for example, a database.
Okay.
I'm using Postgres database.
If I'm using a Postgres database, a managed offering out in the cloud.
Okay.
Let's just say RDS.
Okay.
during development I can run a Postgres container and yeah, it's a smaller
version and it's not a managed offering.
and that works because all the binary protocols are the same.
I can basically just swap out.
My connection URL, and it just works.
But models are a little different.
And so, I've seen some folks that have been like, I'm going to use a
smaller version of a model during local development, but then when I deploy,
I'm going to use the larger hosted model that my cloud provider provides.
Okay, so maybe it's a parameter change, I'm using fewer parameters
so it can actually run locally, and then use the larger version.
The analogy of, okay, me running a local PostgreSQL container and me
running a local model, if it's a different model, it's a different model.
And the results that you get from that model are going to vary quite a bit.
And so that's one of the things that we have to kind of keep reminding
people if you use for example a four billion parameter, So, we're going to
talk a little bit about how you can use this model during local development.
Yes, it can fit on your machine, but then if you deploy and you're using
the 32 billion parameter version in production, those are very different
models and you're going to get very different interactions and different
outputs from the models there.
So, it's something to keep in mind as folks are looking at
building their applications.
So, where are we seeing folks use this?
Well Of course, if you can use the same model across your entire
software development life cycle, then that works out pretty well.
but we're also starting to see a little bit of a rise of, using basically
fine tuned use case specific models or folks training their own models.
and using those for the specific application.
and again, that's tends to be a little bit smaller, more use case specific.
And then, yes, that makes sense to use there.
okay, I need to be able to run that on my own.
et cetera.
again, I think a lot of folks are still feeling out the space and
figuring out exactly how should I think about how should I use this?
and of course, the tooling has to exist kind of before you can actually
do a lot of those experiments.
So, in many ways, we've been building out that tooling to help
support that experimentation.
but I think in many ways, folks are still figuring out exactly what this is going
to look like for them going forward.
Yeah.
I'm trying to imagine what enterprises are doing and building out and I'm imagining
it's not like this, but one of my imaginations is reminding me of 20 years
ago, buying a Google box, which I don't remember the name of, they called it, but
it was this appliance you would put in your data center that was, it was yellow.
Racked
search appliance.
you go.
Yeah.
And I don't know if Nirmal, if you had any customers back then with, if you
were a consultant back then, but that,
I can't name those customers.
so I can, I was in, the city of Virginia beach running the IT and
the data center there, or at least running the engineering groups.
And those were back in the days where we didn't want Google indexing
our internal infrastructure.
You don't want your internal data to be.
accessed or used potentially by this big IT conglomerate.
Yeah.
And so you bring, they, they sell you a, on prem box and you put it in your
data center and it would scan and have all access to everything you could give
it, back when apps didn't really have their own search, Google was providing
that for us, and they would index our email and index our file servers
and databases if we wanted to give it access to them I'm not sure that's
around anymore, but, at a time, Google wasn't going to give away their software
and we didn't all know how to run it.
And, I'm sure it was running on Linux.
And at the time in the mid 2000s, we weren't yet Linux at the city.
For different reasons.
This kind of feels like the same moment for enterprise where.
They're going to have to buy GPUs probably for the first time if they're
going to run it on prem, they're going to want to keep it separate.
They're going to either buy something for the first, probably GPUs if they're
not training models, they just want to run things internally to access their
internal data, or maybe they're doing it in the cloud and Nirmal's company,
AWS, is providing them the GPUs and then presumably because they're getting
dedicated hardware, they won't have to worry about OpenAI or Anthropic
having access to all their stuff
so with respect to ModelRunner and again, just a reminder, these are
my own opinions and not that of my employer, Amazon Web Services.
There's still a lot of use cases.
what Michael was talking about with respect to choosing lots of different
models for different types of tasks, I think there's probably a hybrid model
at some point where folks are using different fine tuned niche models
for specific tasks that they're doing locally, and then as hardware improves,
and hopefully, you know, maybe your 3 year old developer laptop that you get
at your corporation has enough, or the models get, you know, Get, optimize it
enough that they can run on CPU or the GPUs that you have on a corporate laptop,
but there'll be some tooling, probably embedded into the development tooling
and, or you can choose your own models.
and then those, there'll be other models where you need to reach out to the
hyperscalers for those that just you're not going to get the depth of reasoning
you're not going to get the depth of knowledge that you will from something
like Claude in the cloud versus something like deep seek running a quantized deep
seek running on your mac but again this is all changing very very fast and Right
now, we're in a different state where folks have to spend money to access
those larger models, which hasn't been the pattern with respect to software
development in a long time, right?
Not everyone has that advantage, right?
I mean, if you want to use Claude Code and, not hit any major limits, then
you got to pay 200 a month, which, not everyone's going to be able to afford 2,
400 a year, to do software development, there's also edge use cases, right?
So IoT devices, just trying to figure it out.
Also just kicking the tires.
I think that's probably the main, like Michael said, I think that's
just, everyone is just trying to kick the tires as cheaply as possible.
model runner feels like a gateway, like it feels like a gateway drug to get
me Hooked on like the idea, cause I, I mean, I wasn't paying attention to, I
wasn't a machine, I wasn't an ML person or I wasn't building AI infrastructure,
but Docker model runner and, well, and, you know, to a lesser extent, Ollama,
Ollama always felt like it was more for those people that were doing that,
but bringing this capability into a tool that I'm already using actually
felt like, Okay, this is meant for me now, like this is, I don't have to be,
I don't have to understand weights and all the intricacies of how models are
built and work and the different, I don't have to, I don't have to understand
the different file format and whether that works on my particular thing.
it just all kind of works.
It's Docker easy at that point for me.
think that's the key point here of just, let's try to increase the access
to these capabilities and let folks start to experiment and I think Again,
we, as an industry, are trying to still figure out exactly how to use a lot
of these tools and, okay, what is the right size model for the job at hand?
and so, again, in order to be able to do that experimentation, you
have to increase the access to it.
and so, but that's still kind of, you know, Where we are in many ways.
So did we check off another thing on the list?
about to I'm gonna have to keep saying this We're not a news podcast.
We don't talk about the latest things like in general in tech but you
know, I got to give a shout out to Fireship because I have Between that
and a few other channels, I've really learned a lot this year about models.
And I was, there's this little, diagram of sort of the state of a lot
of the open, waiter open, or free, I'm just going to say free models.
They're free in some capacity, that you can download.
And, I was using Devstral.
We talked about it a couple of times on this show already, which
is a model that came out in May.
I actually had a newsletter on that.
Shout out to, Bret.
news, there's this guy, Bret, he makes a newsletter, Bret.
news, you can go check that out.
and I talked about this, that maybe this was the sweet spot because it was small
enough, you could run it with a modern GPU or modern Mac, and it wasn't the
worst, nothing like the frontier models that we get with OpenAI and Anthropic,
but it was something that was better.
And, that's called Devstral.
And then we had Quinn three just come out, I don't know, a
week ago or something that is
I think it was earlier this week.
Oh, see, time warp of AI,
I think it
three days is like three weeks.
It was two days ago.
Yeah,
Oh gosh.
I always assume that I'm seeing a Fireship videos late, but, yeah,
one day ago, so yeah, that's true.
One day ago, there's a newer model coming out from Alibaba, right?
And it is even better, although it does take more GPU memory, I
that's hard to run like on a laptop.
I don't think you can, so that's a perfect encapsulation of like why Docker model
run is there, but also why it's going to be a mix of models going forward.
so we've, we have an AI that helps you use Docker and get started.
We have a tool that helps you run models locally, and understand that
what's the next step in this journey?
by the way, you go to Docker Hub to look for models, or you can
do it in Docker Desktop, or you can look at things in the CLI.
there's many ways to see the models.
you can pull things from HuggingFace now.
That's pretty sweet.
Can we build, can we make our own yet?
Or do we have that?
We can
so, you can package, but you would have to already have the gguff
file and all that kind of stuff.
So we don't have a lot of the tooling there to help you
actually create the model itself.
Although a lot of folks will use container based environments to do that.
we don't have any specific tooling around that ourselves.
So is there, you're saying there's a doc, oh, there, there is.
Oh, I didn't realize.
So there is now a Docker model package CLI for creating, basically
wrapping the GGUF or, other models or whatever, into the Docker OCI
standard format for shipping and pulling and pushing essentially Docker.
models that are in Docker hubs that are in the Docker format.
OCI format.
Yep.
So when we look at agentic applications, step one is, you need models.
It's kind of the brains of the operations.
and then you need tools.
And so you've got highlighted here the MCP toolkit.
That was kind of the first adventure into the MCP space.
And that one was focused a little bit more on how do we provide tools to the
other agentic applications that you are already running on your machine.
So Claude Desktop or using VS Code Copilot on my machine or Cursor, etc.
How do we provide those MCP servers?
And containerized ways, secure credential injection, etc.
Basically manage the life cycle of those MCP servers.
Again, in the use case of connecting them to your other agentic applications.
And so, again, this is kind of where we started our MCP journey.
if you see flipping through a couple of these, actually we just released a
Docker Hub MCP server that allows you to Search for images on hub or, you
know, those within your organization, which is super helpful for like maybe
a write me a Dockerfile that does X.
Well, cool.
Let's go find that the right image that should be used for that.
so again, starts to open up some of these, additional capabilities here.
Yeah.
And inside the Docker desktop UI, there is now a beta tab, essentially,
that's called MCP Toolkit.
And it is a GUI that allows me to explore and enable one of, 141 different tools
and growing that Docker has added.
So like a lot of the other places on the internet that either they host models
like Anthropic or OpenAI, or They're a place where you can create AI applications
and all those places have started to create their own little portals for
finding tools and they may or may not, I mean, most of them all now settle on
MCP, but before we had really MCP as the protocol standard, they were already
doing like OpenAI was doing this before, but they were very, it was proprietary.
You don't know how Evernote or Notion got, showed up as a tool
feature in ChatGPT, it did.
But we just assumed that was their custom integration and now we have this
standard called MCP that everything should interact with everything
properly the way that they should.
At least right now it's the one that's winning.
We don't know whether we'll still be talking about MCP in
five years, but it's here now.
It's what we're talking about now.
And this lights up a lot of capabilities.
In other words, you turn on an MCP tool.
And that sits behind something called MCP Gateway.
So tell me, what is MCP Gateway?
Yeah.
So at the end of the day, the toolkit is a combination of several different things.
the MCP gateway is actually a component that we just open
source at WeAreDeveloper.
So you can actually run this gateway directly.
In a container, completely on its own.
And the MCP gateway is what's actually responsible for managing
the lifecycle of these MCP servers.
it itself is an MCP server.
think of it more like an MCP proxy.
it exposes itself as an MCP server that then can connect to your applications.
But when you ask that server, Hey, what tools do you have?
It's really delegating, or I mean, it's using cache versions of, okay,
what are the downstream MCP servers?
And so it's acting as basically a proxy here.
so when requests come in and say, hey, you know, from the agents at GAP, if
I want to execute this tool, go do a search on DuckDuckGo at that point, the
MCP gateway will actually spin up the container, that DuckDuckGo MCP server,
and then delegate the request to that container, which then does the search, and
then the MCP gateway returns the results.
So kind of think of it as a proxy that's managing the lifecycle of all
those containers, but also, you know, injecting the credentials, configuration.
it also does other things like actually looking at what's going in and out of
the prompts going through the proxy to make sure, you know, secrets aren't being
leaked or, that kind of stuff as well too.
and we're even starting to do some further explorations of what are other
ways to kind of secure those MCP servers?
You know, for example, a file system one should never have network access.
Cool, so let's, when that container starts, you get no network access,
or, the GitHub MCP server, you know, it's talking to the GitHub APIs.
Let's only authorize those host names that it can communicate with.
So, you know, start to do a little bit more of a kind of permissioning
model around these MCP servers, which is where a lot of people are kind of
most cautious and nervous about MCP servers, because it's, they're completely
autonomous for the most part, and you have to trust what's going on there.
this exact feature is both necessary and also Solomon Hyke's
prediction from a month ago.
He was on this show and was saying that we're going to see all these
infrastructure companies and all these tooling companies that are
going to offer to lock this shit down.
I think it's the quote I have to get from him.
that sounds right.
he compared the origin of containers and how it started with developers.
And then eventually it took it and sort of managed it in the infrastructure
layer and provided all the restrictions and security and limitations and
configuration and all this stuff.
And the same thing's happening to MCP.
Where it started out as a developer tool to empower developers to do all these
cool things with AI that they couldn't do and let the AI actually do stuff for us.
And now very quickly in a matter of months, IT is coming in and saying,
okay, we're going to lock this down.
It's crazy.
You can, you know, your prompts can delete your, drop your databases,
your, as we just saw happen on the internet recently this week.
I want to, on this gateway topic though, it can sound complicated.
And maybe the internals are a little, and there was obviously
code built into this program.
But for those of us that aren't maybe building agents yet, or
like really getting into building apps that use AIs in the app, this
just appears as kind of magic.
Like it, you go into the Docker desktop UI, I enable, I go through the toolkit,
there's all these suggestions, everything from, the MCP server for GitHub itself to
an MCP server that could give me access to Grafana data to accessing the Heroku
API and you're looking at all these things and you're just like, I'm enabling them.
it's like a kid in a candy store.
I'm just going check, check, check.
Yeah, I want Notion.
I want Stripe.
I get them in a list, they're enabled, which means they're
not actually running, right?
they're waiting to be called before the gateway runs them in memory.
That's all transparent to me, I don't realize that's happening.
And if I choose to use this toolkit with Gordon, if I just go into Gordon,
And in the Gordon AI, if I don't want to run a local model myself, or I'm
not using Claude Desktop or something that gives me the ability to enable MCP
tools, I can just go in here and say, enable all my MCP tools, all 34 of them.
I've got get ones and I've got, and so now what that means is the Gordon
AI can now use these tools, which makes this free AI even smarter.
And I can say, is there a Docker hub image NGINX?
I don't know if there is.
Let's see.
I've never even tested this.
So it's kind of a, what could go wrong?
let's use Google live on the internet and see what happens.
yeah, I was just saying, look at it, going out and checking, using the
Docker, the newly created Docker hub, MCB, tool That you had, just, released.
so is this going through a, MCP gateway or is this not with the MCP gateway yet?
when you and Gordon AI flip the switch to say, yes, I want to use the MCP toolkit,
basically what that's doing is, in the Gordon AI application here, it's enrolling
the MCP toolkit as an MCP server.
And so then it's going to ask The MCP toolkit, hey, what
tools do you have available?
And so when you saw that list of tools, that's again coming from the gateway.
Gordon is just simply treating the MCP toolkit as An MCP server, which in
itself is going to launch MCP server.
So that's kind of why I mentioned, it's kind of thinking of it like a proxy there.
Yeah, it does feel like one.
Yeah.
And this, for those watching, like it didn't actually work because I don't
actually have access to hardened images, but, I just wanted to see what it would,
what it'd say, but it, the, the UI
Which is the right answer?
Which is the right answer for
did the right thing.
it didn't expose a vulnerability in the MCP server.
But yeah, so it basically, I can give Gordon AI more.
And I can do a lot more functionality, more abilities to do things without
having to run my own model, without having to figure out Claude Desktop.
But I will say, because I'm in love with this toolkit so much, because I love
this idea of one place for my MCP tools, for me to enter in the API secrets so it
can access my notion and my Gmail but I don't want to have to do that in Claude
Desktop and then in warp and then in VS code and then in Docker and then in
Ollama and like every place I might run.
A tool that needs MCP tools or access to an LLM.
So I did it in Docker Desktop.
I enabled the ones and set them up the way I wanted.
And then inside of my tools around my computer that all support AI MCPs,
they all have now added MCP client functionality that lets me talk to
another MCP, any MCP server that speaks proper MCP through their API.
And in this case, what I've done in the warp terminal, because it does support
MCP, is I just tell it, the command it's going to run is docker mcp gateway run.
And it uses the standard in and standard out, which is one of
the ways that you can use mcp.
And then I suddenly have all 34 tools that we enabled in my docker desktop.
Available in Warp, just as long as Docker Desktop's running, that's all I gotta do.
And then, because Warp is using Claude Sonnet 4, or whatever I told Warp to
do, Docker doesn't care about that, because I'm not asking it to use,
think that, we talked about this it's called bring your own key, I guess
that's what everybody's talking about.
Uh, bring your own key is when you want to bring your own Model to whatever
to access,
you're a key to access the model.
Yeah, but in warp in particular, like this is nuanced, but in
warp, you can't usually, you can't yet access your own models yet.
I think they're going to make that an enterprise feature or something.
But if I open up VS code, I could do the same thing.
If I opened up a ChatGPT desktop, I could do the same thing.
And, Aider or like any of the CLI tools, although anything that accepts MCP
so far, I've gotten to work this way.
And it's been awesome because all these, all these different IDEs and AI tools
all set up a little bit different.
Goose you set up differently than Claude Desktop.
So
Client.
they're all, and all they, all of them have different knobs and, ways of
controlling MCP servers and at varying degrees of control and flexibility.
So this is really nice because then you can also just have
all those tools running if you
right.
They look like one giant MCP server.
Yeah, because normally I would have to add each MCP tool as its own server,
I think, it feels like some of the tools are all standardizing on Claude
Desktop as, that config file, which I
the mcp.
json,
Yeah, it feels like that, like everyone's settling on just using that one file.
Which is, I guess it's kind of feels kind of hacky, but I guess it's fine.
it feels like every editor using my VIM settings.
It's like, no, no, no, no.
Calm down.
I don't necessarily want you all to use the same file.
I don't want you all overwriting each other and changing the same file.
so you're bringing up a really important thing, which is, since we're new to this,
the number of MCP servers is not too many yet, even though it does feel like there's
probably like a lot of MCP servers.
I've been using 10 new MCP servers since we've started this conversation.
But it's still like a number of tools that we can still rationalize about.
But probably in another month or two at this rate, there is a limit to how many
MCP tools One single client, I guess you could say, or one instantiation of
a task that you're using like Claude Code or Cline, it's context window can
only use a certain amount of tools.
And so, is there some ideas about breaking up in the MCP gateway, like having maybe
like sets of tools that have like specific supersets or tools or something like that?
Yeah.
Good question.
And so that's a good call out.
And so I actually want to, zoom in on that just a tiny bit there, because
for folks that may be new to this, they may not quite understand that
the way that tools work is basically, it's taking all the tool descriptions.
Okay, here's a tool name.
Here's when I'm going to use this tool.
Here's the parameters that are needed to invoke this tool.
And it's sending that to the model on every request.
And so the model's having to read all that and basically say, hey,
based on this conversation, hey, here's a toolbox of stuff that
I may or may not be able to use.
But as Nirmal just pointed out, like that takes context window there and
granted yes some of the newer models have incredibly huge context windows but
depending on the use case, it's going to affect your speed, it's going to
affect the quality, and so yeah, you do want to be careful of like, okay, I'm
not just going to go in there and just flip the box on all the MCP servers.
Now you have access to everything.
Like, you do want to be a little conscious of that as well.
in fact, I found it funny in a, I was playing in Cursor not long ago,
and you know, they have even just YOLO mode, just go crazy with it.
But even they have A warning once you, I think it's after you enable
the 31st tool of like, hey, heads up, you're getting a little crazy.
So like, I'm like, for the one that has YOLO mode to call me out for being
crazy for too many tools, like it was, it's again, kind of a reminder of just,
okay, you do want to be conscious of the number of tools that you're using.
so to actually answer the question.
Yeah.
It's been something that we've been exploring and kind of waiting to just see
what the feedback is gonna be on that.
Are there separate tool sets that, clients can connect to.
you know, that's certainly a possibility as well, since this MCP gateway is an
open source container, when you run this for your application, not only can
you say, these are the servers I want, but then you can even further filter
through, these are the tools from those servers that I actually want to expose.
So, for example, I think the GitHub official one is up to
72 tools now or something.
It's a crazy number.
but most of the time, I only need maybe three or four of them.
So, I want to filter that.
And that's why you see, Cloud and VS Code and many of these others.
Even though you're pulling in these MCP servers, many of those
provide client side functionality to kind of filter that list as well,
Yeah.
I wonder if we get to a state because the MC, this is getting a little bit
meta, but everything when you talk about agentic AI gets meta really quickly.
So, I wonder if the MCP gateway itself is an MCP server, so it can rationalize
about itself, I wonder if we get into the pattern of, okay, there's this new
task that I want this agent to do, and the first thing, after it comes up with
its task list, the steps it wants to take, is go through that list and then,
ask the MCP gateway to reconfigure itself on each task and turn on Only
the ones that it identified as likely the ones that it needs for that task.
And just dynamically, at any given time, don't have anything
more than five running.
So figure it out.
You can choose whatever five you want, but only have five.
We've done some experiments with that, not quite to that full dynamicness,
but I've even done some ones of a Okay, here's a tool to enable other
tools, is basically what it is.
And, okay, and give me parameters of, okay, do you need, GitHub?
Do you need, Slack?
You know, tell me what it is that you need, and then I'll
enable those specific things.
And then what's cool then is, as part of the MCP protocol,
there's also notifications.
So the MCP server can then notify, The client says, hey, there's a new
list of tools available, and then the next API request to the model
then has this new set of tools.
I
think we're almost
capability is there, but,
I think that's likely the next step.
but it's also kind of like a, yeah, how do you safeguard that?
So it's, Yeah, it's an interesting time period, for sure.
we got an interesting question, is MCP Gateways intent to replace an
API Gateway or in parallel to it?
Great question.
Michael, you want to take that one?
yeah, great question.
I'd say in some ways that there's similar functionality, but they
serve very different purposes.
So an API gateway, I'll just take the most basic example, but I know
there's lots of different ones, An API gateway, single endpoint, and I may
have lots of different microservices.
Let's just pick a catalog.
Okay, so for product related ones, it's going to go to this microservice.
Users, it's going to go to this other one.
Cart, another service, whatever.
And the API gateway is routing all those different requests and rate
limiting, etc. In many ways, like this MCP gateway serves in a similar fashion
in which it's going to be routing to the right MCP server to actually
handle the tool execution and whatnot.
But again, it's only for the MCP protocol.
So it's not going to be replacing an API gateway because it's not doing
normal API requests, etc. It's only for MCP related workloads and requests.
different protocols at play here.
I think that's probably the best way to describe it.
otherwise, you could also say that MCP and API Gateway are likely
going to be running in parallel.
and so probably what I would see would be, I have an API gateway that routes
a request to an endpoint, and then that particular application, let's just say
it's an agentic application, can then have its own MCP gateway to satisfy whatever
agentic flow it needs to use there.
I wanted to, while you guys were having an awesome conversation, I was trying
to draw up, just a visualization to try to represent, okay, so just so
people understand, because this MCP, we could make a whole show on MCP
tools, honestly, from an infrastructure perspective, how do these things talk?
How do they integrate?
The fact that you're talking about that they're just really adding to
the context window is a fantastic fact that, A lot of people could go
months or years using MCP tools day to day and never know that, right?
a normal non engineer could use MCP tools, not understand how
these things are all working.
for those that are into this, are playing around with MCP tools elsewhere
and understanding a little bit of MCP server functionality and client versus
server versus host and all that stuff.
Before Docker's gateway, the MCP gateway, you would have like an MCP client
That whether it's your IDE, your terminal, or AI, chat, desktop, or whatever you've
got, that is acting as an MCP client.
Assuming it supports MCP servers, you can add them one at a time.
So I would add GitHub's MCP server, then I would add DuckDuckGo's MCP server.
I might add Notion's MCP server, since I'm a big Notion fan.
And each one of those servers has One to infinity, tools, which
are, I look at as like API routes.
and each one has its own very niche purpose.
depending on the tool, and this is part of the frustration with the ecosystem
right now is we're only months into this, but it's amazing that all these tools are
all starting to support each other, tools have different ways where you manage this.
Some of them you can disable and enable specific servers.
Some, you can actually choose the tools individually, which
is like choosing API routes.
And to me, it's you're always trying to get down to the smallest amount of
tools that you need to prevent confusion.
Cause I'm, my biggest problem is I enable all the tools because
I get tired of managing them.
I just want them to work.
I just want them all to work when they need to work.
And then I, so I enable them all.
I end up with 50 plus tools.
And then when I'm asking AI to do things, it chooses the wrong tool because I
wasn't precise enough in my ask to trigger the right words that are written
in the system prompt of that MCP server.
So actually, maybe an easier update might be to put another layer on top of the
MCP server and kind of an in between.
so I'm connecting the MCP gateway now to multiple other MCP servers.
So I get, yeah, you're right.
I need another layer here.
That's actually MCP servers.
so, there's now this gateway in the middle, and it, the only negative of this
approach is for right now, because we don't have this futuristic utopia yet,
is that to my terminal, or my IDE, it all looks like one giant list of tools.
And in one MCP server, which is just the nature of proxy, right?
But behind one IP address is a whole bunch of websites, like
you don't realize it, right?
So it is, the analogy still works, I believe, there.
But in this case, because it's connecting all of them together into one proxy,
and the nice thing is, it's, I can see in the memory usage and the containers.
In fact, when Michael was on weeks ago, We saw the MCP gateway spinning up
servers dynamically and then shutting them down and you could see the container
launch run, you know, run the curl command or whatever, and then close.
And it was so quick, we couldn't, capture and swat toggle windows
to see the tools launching.
And, I mean, it's beautiful.
It's exactly what containers were for.
It's, it's ephemeral, it's wonderful.
But, if your IDE or if your chat desktop or whatever is acting as your MCP client,
the agent thing, if that doesn't let you choose individual tools, then this
approach is a little hard because the only way from my IDE that I can, I have to
turn off all of Docker or none of Docker.
It I think.
This gets us to a conversation of eventually we will have this.
I'm thinking of it as like the model, the plan model before the model that will
go, okay, you used all these keywords.
I'm going to pick out the right tools and I'm going to hand those
off to the next model, which is going to do the actual work.
That's probably already here.
Solomon predicted it a month ago.
Yeah, I'm sorry.
What?
so that's what Michael and I, while you were drawing this
Oh, is that what you were just talking about?
that's what Michael and I, we were talking about.
So the gateway itself has its own MCP server that controls itself.
And so we're a few months away from exactly what you were just talking about.
Bret, because of context windows, because there's too many tools, because
of all the things that you did, all the challenges you just mentioned, Bret.
the first step might be the client going to the MCP gateway, MCP server
first and saying, hey, these are the things I'm about to go do.
Out of the list, check your MCP gateway and tell me the list of, MCP
tools that I actually need for that.
And then only turn those on for the next task.
Yeah.
and then it'll just repeat that cycle again.
and then winnow down that list of MCP tools to the only things that
are needed for that task at hand.
So there's another layer here, which we're, and Michael and I,
we were discussing while you were building that beautiful diagram.
It's, people are experimenting with that.
All the pieces are in place, but that this pattern isn't quite there just yet, but
it will likely be, I'm pretty sure this is what we're going to be doing pretty
Nobody wants to go manually choose every MCP server that they're going
to need before every AI request.
almost feels like it takes away the speed advantage of using the
MCP tool to go get the data for me.
if I have to do all this work in each tool independently, Because I often
will have an IDE accessing AI, acting as the MCP client, and then I'll have
a terminal acting as an MCP client.
At the same time, I've got ChatGPT desktop running over here, also
while VS Code, I think a lot of us, eventually evolve to the point where
we've got two or three tools all at the same time managing MCP tools.
We've got, I guess we have multiple IDEs, I should say.
and trying to understand how all this comes together is only interesting right
now, but in six months, we're not going to want to be messing with all this stuff.
We're just going to want this part to work so we can work on building agents,
All right.
So Compose, my favorite tool, a lot of people's favorite Docker tool, other
than the fact that Docker exists.
you announced at, we are developers that Compose is getting more.
There's functionality in the YAML specifically, where I guess we're talking
about the YAML configuration that drives the Compose command line, that in just
three months ago, you were adding model support, and that was like an early
alpha idea of what if I could specify the model I wanted Docker Model Runner
to run when I launch my app that maybe needs a model, a local model, and I
use the example and I have a An actual demo over on GIST, that people can
pick up that you simply, you, you write your Compose file, you use something
called open web, open, what is it?
Open web, web, open web UI, I think.
Yeah, horrible name.
Extremely generic name for what is a ChatGPT clone, essentially.
the open source variant, which can use any models or more than one model.
It actually lets you choose it in the interface.
And all you need is a little bit of compose file.
So, I created a 29 lines, and it probably needs to be updated because
it's probably outdated, but, 29 lines of Compose that's half comments that allows
me to spin up an open web UI container while also spinning up the models or
making sure, basically, that I have the models locally that I need to run it.
And this gives me a ChatGPT experience without ChatGPT.
Thank you.
And you guys, you enable this.
Now you're not creating the models.
You're not creating the open web UI.
you're simply providing the glue for it to all come together
in a very easy way locally.
Yeah.
as we agentic apps need three things.
They need models, they need tools, and then the code that glues it all together.
What the Compose file lets us do now is define all three of
those in a single document.
here's the models that my app is going to need for MCP gateway that I'm just going
to run as another containerized service.
And then the code, the custom code, can be really any agentic framework.
this example is, Open web UI, but that Compose snippet what we've done is
we've evolved the specification now models are a top level element in the
Compose file, which is pretty cool.
This just dropped in the last couple of weeks, so this is brand new.
Gotta update my gist.
yep, and so where before, yeah, you had to use this provider
syntax, and that still works.
now it's actually part of the specification.
defining a model, This is going to pull from Docker Hub.
again, You can have your own models and your own container registry.
It's just an OCI artifact.
You can specify that anywhere.
then we've got the services,
And then the app itself.
What's cool about the model now is with the specification evolution,
you can now specify, hey, this is the environment variable I want you to
basically inject into my container.
to specify what's the endpoint, where's the base URL that I
should use to access this model.
And then what's the model name as well.
So the cool thing then is I can go back up to the top level model
specification, I can swap that out and the environment variables will be
automatically updated and as assuming that my app is using those environment
variables, Everything just works.
So again, think of Compose as, it's the glue that's making sure that
everything is there for the application to actually be able to leverage it.
Yeah.
the gateway part here was pretty cool to me.
That I can add in my tools, my MCP tools inside of the YAML file.
when I saw that part, I was like, yes.
that is like my vision, my dream is that I can pass a composed file to
someone else and it'll use their keys.
Presuming, my.
Team is all using the same provider that we would have the same.
Because open, well, open AI, base URL, open AI model, and
then open AI API key or whatever.
if you're going to use ones in the SAS, like those are all pretty generic.
Even if you're not using OpenAI, they're all pretty generic, environment variables.
So I guess this would work across teams or across people
well, and that's a good point to call it.
One of the things that OpenAI did when they released their APIs was basically,
hey, here's a specification on how to interact with models that pretty much,
Everybody else has adopted and used.
and so the Docker model runner exposes an OpenAI compatible API.
And so that's why you see these environment variables kind
of using the OpenAI prefix.
Because again, I can use now any agentic application that can talk to
OpenAI or use the OpenAI libraries.
And it's just a configuration change at this point.
All right.
Now, the coup de gras.
Piece la resistance.
I can't even do my pretend French, All this stuff has been running locally.
Like when we think of Docker desktop, we think of everything locally.
And then a year or two ago, Docker launched, Docker build cloud, which was
like getting back to Docker's roots.
I almost feel like of providing more a SaaS service that essentially is.
Doing something in a container for me.
And in that case, it was just building containers using an outsourced build kit.
So it was better for parallelization and multi architecture.
it was sweet.
And I love it for when I need to build like enterprise tools or big business
things that take 20 minutes to build.
None of my sample little examples do that, but anything in the real world
takes that long and you need to build multi architecture and generally it's
going to be faster in a cloud environment.
So you provided that.
Now it feels like you've upgraded, like it's beyond just building, and it does any
image or any container I want to run, any model I want to run, I guess, not maybe
any, I don't know if there's a limitation there, but, bigger models, then maybe I
can run locally, and then, also builds.
So it can do building the image, hosting the container, running the
model endpoint for a, QIN3 or whatever.
I can now do all that in something called Offload.
So tell me about that.
Docker Offload, the way I explain it to people is, hey, you need more resources?
Burst into the cloud.
And so it's basically, I'm going to offload this into the cloud,
but yet it's still, everything still works as if it were local.
So if I've got bind mounts, okay, great, we're going to automatically
set up the synchronized file shares.
And so all that's going to work the way using mutagen and some of the other tools
behind the scenes to make that work.
Port publishing that still works as you would expect it to.
So again, it gives that local experience, but using remote resources.
I'm just offloading this to the cloud, but yet it's still.
My environment.
and so, yeah, to make it clear, like, this is not a production runtime
environment, I can't share this environment out, or, I can't, you
know, create a URL and say, hey, check this out, colleague, or whatever,
it's still for your personal use.
Now, of course, can you make a, Cloudflare tunnels, and I'm going
to make it production, sure, but I
wouldn't
could hack
that.
but yeah.
Yeah.
So what is the intent?
so what is the use case?
the big
should I use Docker offload for first?
Yeah, so, okay, great.
You're wanting to play around these agentic apps and, you know, we
were talking about not everybody has access to high end GPUs or,
you know, M4 machines and whatnot.
great with the flip of a switch, and you had it there in Docker Desktop,
but at the top you just flip a switch, and now you're using offload.
and so now you've got access to a pretty significant NVIDIA GPU, and additional
resources, and so yeah, as you're, We see the use case, especially more for
the agent applications, because that's where those resources are needed.
It does open up some interesting doors for, maybe I'm just on a super lightweight
laptop that I'm using for school and I don't have the ability to even run
a lot of my containerized workloads.
Great, I can use that for offload, you know, offload that to the cloud.
it does open up some interesting opportunities for, Use cases beyond
agentic apps, but that's kind of where the big focus is right now.
So if you're like a Docker insider, or if you're someone who's used Docker a while,
it's the Docker context command that we've had forever, augmenting or changing the
environment variable docker underscore host, which we've had since almost the
beginning, and it allows you from your local Docker CLI, And even the GUI works
this way too, because I could always set up a Docker remote engine and then
create a new context in the Docker CLI.
That would use SSH tunneling to go to that server, and then I could run my Docker
CLI locally, my Compose CLI locally, and it would technically be accessing and
running against, the remote host that I had set up, but that was never really a
cloud service, like it was never, no one provides Docker API access as a service
that I'm aware of, and, the context command, while it's easy to use, and you
can actually use it on any command, you can use docker run dash dash context,
I believe, or docker dash dash context run, I can't remember the order, but,
you can change that on any command.
these are all things that existed, but you made it, stupid easy.
It's just, like you said, it's a toggle, it's so easy.
You just click that button and then the UI changes, the
colors change, so now you know.
You're now remote.
Yeah, and so I'll go ahead and say too, behind the scenes, it's using context.
It's using those exact things.
The tricky part is, because I've done similar, development environments where,
I'm going to work against a Raspberry Pi at home or, whatever else it might be.
the tricky part is when you want to get into bind mounts, file sharing
kind of stuff, or port publishing, and I want to be able to access
that port from my machine, like automating all those different pieces.
That's not trivial.
I mean, it's possible.
a separate tool, yeah, you gotta download ngrok or something,
And so this brings all that together into a single offering here.
That's pretty amazing.
Like there's a lot going on underneath the hood that, switch is hiding
a lot of different functionality.
Like
it's.
To make that very transparent
And this supports builds too, right?
So like.
When I toggle this in the UI, or is there a CLI toggle?
Yeah, there is.
okay.
So if I toggle this, it's, yeah, you're like, you're saying it's a context
change, but it's UI aware, and it takes in all the other little things that we
don't think about until they don't work.
And then we're like, oh, yeah, it's not really running locally anymore.
So now I can't use localhost colon.
Well, that all just, I'm going to show you how this works and you don't even
have to know kind of like the rest of the Dockery because you don't really
have to know how it works underneath.
but if you think it's too much magic, I like to break it down and say,
it's just really the Docker context.
I don't actually look anything at the code.
I don't know really how it's working.
But to me, when I went and checked, it does change the context for me.
It actually injects it and then removes it.
I did notice it from the CLI.
I could change context.
And it would retain the context, but if I use the toggle button, it deletes
the context and then re adds it.
Regardless, it is in the background, it's doing cool things.
I think the immediate request from the captains was, can I do both?
Can I have, Per workload or per service offload so that just my model's remote
and maybe that really big database server and then all my apps are local.
I don't know why I would care, but like it, that's something that people ask for.
I'm not sure that I care that to that level.
I think I'm fine with either or, but I can understand that if I'm running some things
locally already and I just want to add on something in addition, it would be neat
if I could just choose for one service.
Yeah.
So as of right now, it is an all or nothing.
you're doing everything local.
You're doing everything out in the cloud.
there's not a way to.
Split that up yet.
it's something that we've heard from a couple of folks, but
again, it's that same thing of, tell us more about the use cases.
So if that's a use case you have, feel free to reach out to us and
help us better understand, why you might want to split runtime.
hosting, split environment, hybrid environment.
That's the correct term.
Why do you say it like that?
and just to be clear, offload has its own cost.
like this isn't free forever for infinity.
You can't just take up a bunch of GPUs.
I was asking the team a little bit and without getting too nerdy, it
sounds like it isolates, it spins up a VM or there's maybe some hot VMs.
And I get a dedicated.
OS essentially it sounds like so that I can get the GPU if I need.
And you kind of get an option of, do I want GPU, servers with GPUs or not?
do I, am I going to run GPU workloads or not?
And that affects pricing do we get anything out of the box with
a Docker subscription or is it just an, a completely separate
So actually it's a. Kind of private beta, but yet people can sign up
for it and that kind of stuff.
folks will get the 300 GPU minutes, which isn't a ton, but it's enough to
experiment and play around with it.
and then, start giving us feedback, etc.
Yeah, if you spin up the GPU instance and then go to lunch, by
the time you get back, you'll have probably used up your free minutes.
It's a long lunch,
hey, that's my kind of lunch.
but yeah, so we went an hour, and we barely scratched the
surface, do we cover it all?
did we list at least all the announcements of Major features and tools.
I don't even want to say we've covered all the features because there's
probably some stuff with MCP we missed.
So you open source MCP gateway, but we should point out you don't actually
have to know, like you can just use Docker desktop and MCP tools locally.
But the reason you provide an MCP gateway is open source is so
we could put it in the compose file and then run it on servers.
think about it this way.
the MCP toolkit bundled with Docker Desktop is going to be more for,
I'm consuming, I'm just wanting to use MCP servers and connect them
to my other agentic applications.
And the MCP Gateway is going to be more for, now I want to build
my own agentic applications and connect those tools to, Those, those
applications that we're running there.
Yeah.
Do you see people using MCP Gateway in production?
Do you see that as like a. Not that you provide support or anything like
that, but is it designed so that
We've got a couple of folks that are already starting to do so.
stay tuned for some use case stories around that.
Yeah.
Awesome.
well, this is a lot.
I feel like I need to launch another 10 Docker YouTube uploads just
to cover each tool specifically, each use case specifically.
there's a lot here, but this is Amazing work.
I mean, I don't know if you have a fleet of AI robots working for you yet, but
certainly feels like a lot of different products that are all coming together very
quickly that are all somehow related to each other, but also independently usable.
And, I'm having you on the show as usual is a great way to break it down
into the real usable bits What do we really care about without all the
marketing, hype of general AI hype, which is always a problem on the internet.
But this feels like really useful stuff.
Um,
Eivor, another podcast.
I don't know, Eivor, what's up?
Are you requesting yet another podcast?
Um,
a whole new show
about Compose provider services?
Oh, yes.
Also, you can now run Compose directly from, well, you can use Compose YAML
from directly inside of cloud tools.
The first one was Google Cloud Run.
So I could technically spin up Google, which I love,
Google Cloud Run is fantastic.
Um, it would be my first choice for running any containers in
Google if I was using Google.
Um, and so now they're, accepting the Compose YAML, spec, essentially,
inside of their command line.
So this is like, this feels like the opposite of what Docker used to do.
Docker used to build in cloud functionality into the Docker tooling.
But now we're saying, Hey, let's partner with those tools, those companies,
and let them build in cloud or compose specification into their tool.
So we can have basically file reuse.
YAML reuse.
Is that right?
yeah.
So this is the first time exactly in which it's not Docker tooling that's providing
the cloud support, but it's cloud native.
They're the ones building the tooling and consuming the Compose file.
yeah, it's a big moment.
And as we work with Google Cloud on this, yeah, you can deploy the
normal container workloads, etc.
But they already have support for Model Runner to be able to run the models
there as well, it's pretty exciting And I know, the provider services
this is how we started with models.
having support in Compose, where
That was another service in which the service wasn't
backed by a normal container.
the old method.
yes, but what's cool about this is, so first off, these hooks still are in place.
So that a Compose file can basically delegate off to this additional
provider plugin to say, hey, this is how you're going to spin up a model.
But it starts to open up a whole ecosystem where anybody can make a
provider or, okay, hey, I've got this cloud based database, just as an example.
And, okay, Now I can still use Compose and it's going to spin up
my containers, but also create this cloud based container and then inject
environment variables into my app.
again, it starts to open up some pretty cool extensibility
capabilities of Compose as well.
I think we, yeah, we need to bring Michael back just to dig into that because, it's
essentially like extensions or, Plugins
Yeah.
for
Yeah, so Compose is about to get a whole lot more love.
It feels like it's already, I mean, it's been years since we've
added a root extension or like a,
top level,
top level build.
it's not every day that Docker decides there's a whole new
type of thing that we deploy.
Now we have models, we'll see if providers someday become something.
that'll be cool.
and this is all due to the Compose spec, which now allows other
tools to use the Compose standard.
And that's just great for everybody, because everybody uses Compose.
it's like the most universal YAML out there, in my opinion.
great.
Well, I think we've covered it all.
Nirmal and I need, another month to digest all this, and
then we'll invite you back on.
do it.
but yeah, we've checked the box of, Everything Docker first
half of the year, stay tuned for the second half of the year.
I actually sincerely hope you don't have as busy of a second half,
just because it's, these are a lot of videos I got to make, you're
putting a lot of work into my, inbox,
We're helping you have content to create.
I know, yeah, there's no shortage of content to create right now with Docker.
I am very excited to play with all these things.
I sound excited because I am excited.
This is real stuff that I think is beneficial and largely free, largely,
Like almost all of this stuff is really just extra functionality that
already exists that in our tooling without adding a whole bunch of SaaS
services we have to buy on top of it.
yeah, so congrats.
People can find out more docker.com.
docs.Docker.com, dockers got videos on YouTube now they're putting up
YouTube videos, so check that out.
I saw Michael putting up some videos recently on LinkedIn.
it's all over the place.
You can follow Michael Irwin on LinkedIn.
he's on BlueSky.
I think you're on BlueSky.
Um, or, or, where, yeah,
figured out where I'm hanging out.
Thanks so much for being here.
Thank you, Michael.
Thank you, Nirmal, for joining and staying so long.
I'll see you in the next one.