Bret: Welcome to DevOps and Docker Talk, and I am your host, Bret Fisher.
Technically, the full name of this is Cloud Native DevOps and
Docker Talk with host Bret Fisher.
But, you know, DDT is what we call it inside the team.
So DDT, we're now in our seventh year of podcasting, and if you include
the livestreams I was doing before we started the podcast, I think
it's over eight, maybe even nine at this point, which is pretty amazing.
Uh, not a lot of podcasts get to last a decade, so I'm anxious to
get there, and still have it called DevOps and Docker Talk, which DevOps
never died, Docker never died.
Like, we didn't have to change the name
But on here, we talk about all cloud native things, and another Kubernetes
project has begun and has now showed up as a CNCF sandbox project
this year, and that is OpenChoreo.
Hence, open…
Which stands for Open Choreography, and that's where Choreo… I was
actually mispronouncing it as choreo or something with a hard H, but it's choreo.
And this project is all about adding on AI agents and essentially a new abstraction
on Kubernetes itself that interfaces between what we're all now using as our
primary interface to everything software, which is agent harnesses, and what the
Kubernetes API and other APIs provide that are typically on Kubernetes, like
your monitoring or your observability or whatever else you might need in there.
There's a lot that goes on.
But this project is adding a new abstraction layer, I'm calling it an
abstraction layer, because it provides API, MCP, CLI, and web UI all to
front end everything else you typically deal with, whether that's your GitOps
tools like Argo, your orchestration
Your storage, your observability, Kubernetes itself, your
authentication, like all that stuff.
And we are now in the world where we're all playing around with agent SDKs and
maybe trying to figure out what can we automate in our infrastructure safely
with reliable LLM prompts like this.
These are all things we're probably thinking about, right?
If you're not doing it, you're thinking about it, you're waiting for someone
to tell you how to get it done.
And OpenChoreo is a new project that came out of WSO2, the company, and
they were running it on their own SaaS infrastructure and realized that it might
be useful to others, which by the way, was also how the Docker project got started
by a SaaS company or a cloud company realizing that their infrastructure
tooling was maybe the most useful part of their product, and released it for free.
So OpenChoreo came out of a business hosting a legitimate
tool for Kubernetes users and felt like it needed to be open source.
So now they're gonna be running that on their own platform
for customers eventually.
But for now, they're just focused on getting it open source.
And now that it's officially a CNCF sandbox project, that means they've met
the minimum level of sort of maturity to explore whether this could succeed in
the marketplace, explore whether or not those of us that are building open source
tools or running open source tools on top of Kubernetes would be interested in
an agent platform on top of Kubernetes.
And specifically what this is about is about us administrating and managing
these environments with agent harnesses through the OpenChoreo abstraction,
so through their MCP API CLI.
This is not about running inference or building custom models.
This isn't about being an AIOps or MLOps person, and this isn't
about running custom agents that we built in pods inside of a cluster,
although that's certainly possible.
You can do that.
But the main focus here is on exactly what I talk about on this show, as well
as my other podcast, Agentic DevOps.
And quite frankly, this podcast could probably be on both.
I'm trying to prevent that from happening because they're, you
know, we've all got podcast players.
You can add both of my podcasts to the, your player, right?
So you know that that's a thing.
Um, and this one I'm trying to keep focused on CNCF projects, Kubernetes and
Docker projects, and the Agentics DevOps podcast is really about the patterns,
behaviors, and tools for just being a DevOps or platform engineer or SRE
who's using AI to get their job done.
That's what that podcast is about.
Technically, this could be on both.
But, uh, anyway, On the show this time, I have Lakmal and Sameera from
the project team over at wso2 talking about this, getting into the details
of what it does and doesn't do, and also explaining to me how it's not
what I actually thought it was when we first started talking about it.
And we even on the livestream, which you can link in the show notes, there's
links to the actual demos that we tend to cut out of the podcast version of this,
but the demos are from the livestream.
You can see how the product works, how you can use agents to interface with
Kubernetes, which we, could use agents before, but we didn't necessarily
have all of that infrastructure in place besides just a raw API.
So this helps solve a lot of that with a new level of abstraction
that we get into through this show.
So let's get into it Welcome to the show.
in the middle there, we got Lakmal, and on the right, Samira, both from WSO2.
Lakmal, tell us about yourself.
Who are you?
Why are you here?
Lakmal: Yeah, I'm, working as a VP and distinguished engineer at WS02,
also a Choreo, DM before Choreo BU.
yeah, I started my career as a system engineer, in 2000, one,
and last 25 years, I engaging with building many, many platform.
Yeah, here we are today.
Bret: Nice.
Sameera, how about you?
Sameera: Hey, Bret.
I'm good.
I'm Sameera Jaswal.
I'm one of the core maintainers of OpenChoreo at WSO2, too.
I started my career in the middle as a, you know, you can think of programming
language design, compile engineer, platform architect, and then you'll
hear more about how abstractions and compilation throughout this talk.
me.
Bret: a gist by the end of what you really are into.
I'll get it.
Okay.
all right.
So y- you all reached out.
This is a new project.
I'm always excited about new CNCF projects because it kind of… It--
Once they've accepted into the sandbox process, we kind of get an idea.
I feel like it's reading the tea leaves a little bit.
Not everything is a winner, not every project makes it to graduation, but it's
an interesting way, I think, to, like, separate sort of the hobby projects or
the experimentations from things that teams are taking seriously, 'cause
you don't really go for sandbox status unless you're taking things seriously.
That's where I sort of like wake up and start paying attention to
the new projects, and I'm excited to talk about this today because
I'm obsessed with AI agents.
I'm obsessed with, like, how we manage infrastructure with agents involved.
How do we put MCP in our clusters safely?
how do we sit on our harness as our primary mechanism for interacting
with, I don't know, 20 to 40 years of complexity that we've all built up?
Like, the number of abstractions we all have to deal with today is insane,
and it's amazing that anyone can actually go to a conference and have
a common conversation, 'cause there's a thousand tools we're all using.
So I'm excited to get into this.
how did the project get started?
who's gonna pick the first question?
Like, how did this project come out of the company?
Lakmal: Yeah, we started this, software offering called
Choreo SaaS, five years back.
it's, it start as a, the IPaaS initially, like a integration platform as a service.
Then it evolved into the… You can run all other services
within the platform itself.
So it's, it evolved into the internal developer platform, couple of years later.
Name is also coming as a choreography.
It's, we shorten it to the Choreo.
It's like a choreography your microservices in a certain way, like a
Kubernetes do the orchestration, we do choreography for all your microservices.
That, that how the name also came.
so we were offering this SaaS offering to our customers, enterprise customers.
It's fully managed by WS02.
we were running Choreo version 2, and two years back we started
Choreo version 3, project.
So next version of the, our current offering.
Then eventually it's become the OpenChoreo and in WS02, all our software open source.
So we thought of, "Okay, why not we are running… we having this software
running as our SaaS. Why not open source it?" That's how this OpenChoreo, started.
Then, this year, January, we have donated into the CNCf, Foundation.
Now we are a sandbox project.
Bret: Yeah.
All happening so fast.
that's a similar path, by the way, to how Docker was invented, 'cause it
was a platform company saying, "Hey, this piece of our platform is pretty
interesting, but it's also c- at this point maybe a utility. let's just give
it away." Turned out that that worked.
so it's not a bad idea to do this.
what was the unique problem we were trying to solve with this?
Like, I say we.
I'm, I'm a part of the team now.
how is it, how does the team look at it from, like, this is a unique
thing in a world where we've got dozens of Kubernetes distros?
In fact, first off, I should say, do you even call, do you call
this a, do you think of this as like a distribution of Kubernetes?
Sameera: N-not really.
Bret: No?
Like, do you see it as like a vanilla Kubernetes with some things bolted on?
Or like how is this different from like a Rancher?
Sameera: okay, I'll, I'll take that question.
in my opinion, so OpenChoreo is something you, something we
have built on top of Kubernetes.
Bret: Mm-hmm.
Sameera: So you can use any of Kubernetes distro, Rancher, EKS, AKS,
like any Kubernetes, and then you can just install OpenChoreo on top of it.
So the idea behind is that, so you have Kubernetes and all the
other tools, and then, so you have a developer platform, I think.
what we have seen in the industry is, so you ca- you have this developer
platform with Kubernetes and Argo CD and all the things, glued on,
and then you expose that layer to developers, and you build a developer
experience around it, portals, MCPs.
then we have figured out that, that way you expose the platform
complexity to your developers.
So they'll have to learn Kubernetes, they'll have to learn all the tools.
I think what's unique about here is, I don't know whether this will work out, we
build a middle layer, abstraction layer on top of Kubernetes and other tools.
And that's what Lakshman said earlier.
Through like six years, we have figured out the abstraction layer.
That took some time for us to build.
And then that abstraction layer helps for us to build a developer experience layer.
So developers, they're like kind of abstracted away from Kubernetes, per se.
They can still see what's going on, but they may not to learn a lot of things
about Kubernetes and other tools.
Bret: Yeah
Sameera: so that's about one of the unique layers in the platform.
Bret: So when we talk specifically about Open- OpenChoreo,
is it installing all the parts of this?
Okay, so we're looking at it for the, for the audio audience.
We're looking at a visualization.
I love diagrams 'cause they really help me I l- the conceptual-ness of like what a
thing is, is sometimes instead of a bullet list, I love these, so, I brought that
up, and there's all sorts of com- It's…
To me, it looks like a full-fledged, fully built out Kubernetes cluster
with all the extras, right?
Like, all the necessary things that a sort of an enterprise cluster would have.
it's got these data components.
You know, you're mentioning Keda for scaling.
You've got Cilium in the networking.
You've, you've got, you know, API gateways in there, and then you
showed part of the diagram, there's the workflow part where it's Argo
workflows and build packs in there.
And then you've got observability layers where you've got probably , I don't
see Loki in there, but like Prometheus and all the OpenTels, OTel stuff.
And then over to the side, there's OpenChoreo as this control plane.
So it's almost like to me it's like the control plane on top of the control plane.
so is this really just focused about those things on the left and you're showing how
it interacts, or is this actually helping us implement the rest of this puzzle?
I was a little confused by that.
Lakmal: Yeah.
I think the main idea behind the, having the unified control plane
across all these, capabilities.
p-platform engineers or developers or even now AI agents can interact with
control plane via our different protocols.
And then control plane do the, all the orchestration across all these other
planes, like you, you mentioned the data plane with the Cilium network policy,
with the S-Scale Zero, with the Keda.
All these thing can orchestrate using a, a control plane with the governance
enforce through the control plane.
That's the main idea behind.
So what, what have seen as a problem, even we expose all these services to
the end user, it can be a platform engineer, it can developer, then
you can't enforce the governance.
But having this single control layer, we can have the guardrails,
policy enforcement, and y- single, pane of glass, we can see
everything within the control plane.
So that help to orchestrate across entire different pl-planes.
That planes, built b- using the modular architect in OpenChoreo.
So for example, i-if you take a API gateway, it can be come from the, WSO2
gateway, K gateway, so, Co-Kong gateway.
It doesn't tie to a single vendor.
So it, it's building an entire ecosystem around it.
it's not only the platform itself.
Users can pick and choose what they want to use in their data plane.
so that, the abstraction layer help to, os- orchestrate all these
different, vendor tooling that they want to use in the, their planes
Bret: so, so is it-- If I'm a platform engineer, I-- and I'm listening
to this show, that probably means I have platforms today, right?
Like I've already got Kubernetes in production.
I've already got a lot of these components.
So is the idea h-here that this particular project is for implementing the left
side of my screen of the specific OpenChoreo control plane components,
and then including those system components that would link or integrate
me with the APIs of those other things?
Is that how-- Is that sort of where you're drawing your boundary
of where your project ends?
Sameera: Yeah.
yeah, I think that- that's a good way to put it, Bret.
So our main component would be the control plane, that you
see on the left of the screen.
And then at the same time, there are certain system components
that are running in all the other planes as well, data planes,
observability plane, workflow planes.
So if you're already running these projects, then it's just a matter
of like installing our system components and organizing your sys-
platform architecture in this way,
Bret: Yeah
Sameera: right?
and then I think what Lakmal said about abstractions is that the…
When you think about these developer platforms, primary users are platform
engineers, as you said, and then, primary consumers are developers,
Bret: Right
Sameera: So, so they will perhaps, they will use the experience plane,
UI, MCP server and things like that.
They will say, "I, I want a project, I want a component. I want component to
de- deploy on dev environment," right?
And the control plane understands that and executes and compile that into
Kubernetes and other projects, so in a way that other projects understand, right?
So that's how these arrows and everything, that's the idea behind these arrows.
Bret: Yeah.
And when I'm looking at this diagram, I mean, from s- from sort of a
platform team perspective, I mean, I recognize all the names, right?
It's a lot of terminology, from different, CNCF projects.
But, are you thinking of this as AI is the sort of main interaction point?
I see here that we've got, like there's, I think there's a, a web console or
something, So, do you think about this from the perspective of like AI
is going to probably be increasingly the way we interact with Kubernetes?
So this thing, you know, we've got these arrows in here of how things are
interacting with the Kubernetes cluster, and it feels like it's all going through
the OpenChoreo control plane to get there.
'Cause you've got this list of like, we could use the U- Backstage UI,
presumably that would be for humans only.
unless maybe there's some pretty graphs for the AI to look at.
I'm not sure why it would use that.
But then you've got MCP, CLI, and API as these other interaction endpoints, and
they're not… When I think, when I see this diagram, I think, "Oh, this allows me
to put another layer of abstraction for my AI to work through, and so I'm not running
my AI directly against a kubectl API, or, you know, the Kubernetes native API."
I've sort of got this enhanced layer, with additional functionality, presumably
maybe some… Yeah, I see you have authorization and access control,
which is a big thing for me lately.
Like how do we minimize the blast radius of giving agents access to different
tokens and like not giving them root, you know, God rights to the cluster every day?
which I see a lot of people doing by just giving them the kubectl admin abilities.
is that kind of the, the goal of, of where you see agents being
responsible for all these clusters?
Lakmal: Yeah.
E-exactly that, you describe is correctly, Bret.
So, what we want to have is, now in the developer platform, we give the
golden path, the guardrails, the policies to our human developer.
The same way we want to expose this golden path to the AI agents
that interact within the platform.
So in, in OpenChoreo, the agents are in first class citizens, so
they can, at the moment, current, release, we exposing two MCP servers.
One is the OpenChoreo control pane MCP server, other one is the
OpenChoreo observability MCP server.
So even external agents can interact with this MCP as well
as internal agent, inbuilt agents can interact with MCP C- servers.
Now, when they interact with MCP servers, they have the permissions
and all the guardrails will apply same as the developer, using the platform.
It's a unified experience, whether it's a human developer, human SRE, or a agent,
working with the, SRE a- or as a developer
Bret: Very cool.
All right, so this shows up as a whole bunch of MCP tools that I
could plug in locally to my local… So if I've got Claude Code running,
I plug the MCP endpoints in.
do you know, like, the estimated tool count?
Like, what are we dealing with here?
100, 100 tools?
20 tools?
Lakmal: yeah.
we have, depending on the user persona interact with MCP.
So we have the developer persona, we have the SRE persona.
They give different, tooling based on the, what their role.
Yeah, so based on that they can do different, different, activity.
Sameera: roughly 100.
Bret: Okay.
Okay.
Sameera: say 100 tools altogether
Bret: a not insignificant amount of tools, but that's a good point.
So what you're saying is based on who I'm au- authing as, which role I'm authing
as, I, I get a scoped set of MCP tools based on my permissions essentially?
Yeah.
Sameera: exactly
Bret: So, all right.
so these agents are coming, you're listing AI agent modules here.
You got SRE, FinOps, and architect.
are these things that are, that are coming defined?
Like, and what does that agent mean to me?
Like, if I'm on my local harness, does this mean that those are long-running
agents that are sitting in the cluster?
Like, how do I interact with those agents?
Sameera: like Lakmal said earlier, we have internal agents and external agents.
They both use the same experience, plain MCP servers, tools.
Claude Code is the external agent.
these FinOps, SRE, and architecture agent, they're like platform
agents running in the cluster,
Bret: Yeah
Sameera: and they are reacting to certain events in a way.
For example, if you, if there's an alert, like lot of i00, errors are
going on, then our SRE agent will react, and then it'll give you some report on
what's going on, why this is happening.
So you can configure that way.
You can configure the agent with certain alerts, types of alerts,
memory, memory, like if you are consuming high memory, certain limit
that SRE agent will again react.
Bret: Sure
Sameera: FinOps agent is also similar in that way.
Yeah.
Bret: I see.
So each of those agents comes sort of predefined, it sounds like.
Like, they have a scope to them.
And then presumably, I guess I'm somehow through the control plane, I'm
plugging those into, like, my messaging apps or other… Like, into Slack or
whatever, and that those are how those agents reach out to me is through…
Or do they go through a middle layer of, like, an alerting system
like Prometheus has for alerts?
Like, how does that path look for the agents letting
us all know of the problems?
Lakmal: Yeah.
at the moment it's going a middle layer, the alerting system.
we have the log-based alerting, with the OpenSearch and other layer,
and the Prometheus for the metrics.
So agents also ke-going the same layer.
this alerting system can be, configured with the PagerDuty or
Slack channel or the other channels.
That how the architecture currently working.
Yeah
Bret: Yeah.
And so this feels like it's designed to be sort of, I mean, we, I think the website
or something said batteries included.
I think I included that in the, in the thumbnail.
It, 'cause it, a lot of the, the conversations that I have,
I mean, I'm lucky to be in this, this DevOps, Agentic DevOps guild
that I run, and we meet weekly.
So I get to talk with dozens of teams around, like, what they're trying to
do right now in terms of everything agents related to infrastructure
and this exact stuff, right?
So people are sort of in that middle mode right now where
we've-- the local agent harness has kind of become common knowledge.
Like, we've all experimented with GUIs and TUIs and multi-agent
management and sessions and skills.
Like, we're all, you know, this is the year of s- playing with skills.
And, but then the minute that I have to start running agents, I'm, I just
call these things server agents b- to try to define the difference.
These 20, these long-running, always looping agents that are sitting there
checking something or, continually polling for something or waiting for a
webhook, maybe if they're event-driven.
Like, these things are probably gonna be everywhere.
We're gonna have them all over the place.
And y- you're providing th- a few of those that are sort of out of the box defined
to do specific things, so I don't have to go find an agent SDK and make up my ideas
of what an agent might do in my cluster.
'Cause that kind of feels like what a lot of us are doing right now,
is we're sort of making up, we're writing little a- agents everywhere.
We're making it up as we go along 'cause we don't really know what
parts that they, we would automate or which parts they would be good at.
You know, what parts of summarization and human judgment do they do well,
and what context do they need?
And that's… So, but it feels like the rest of this system, is
the rest of this system to help them with the context problem?
'Cause that always feels like a really hard problem right now around
how, what context do I need to give my agent to make it useful and not
hallucinate, and how do I get at that stuff at the time it needs it?
Lakmal: Yeah, exactly.
So I think the main idea behind we have this ecosystem, within
the ecosystem we have the agents.
So, in community or even, we, OpenChoreo maintainers will release a agent, then
our users can pick and choose this agent run in their OpenChoreo system.
So for example, for SRE agent, the intention behind that, okay, when I say
when something happen, SRE agent can, triggered and we feeding, all the logs,
matrices, config changes, code changes to the SRE agent within that window.
Then, what happen, so it can generate the RCA report, root cause analysis report,
and also alerting to the SRE, human SRE.
When they come into the system, they already have the root cause
analysis report there, that is providing to the SRE agent.
And also we are not stopping there.
this agent can provide the re-remediation action as well.
Okay, this is a quick fix if you want to, fix this issue.
So then at the moment it's a human in the loop, SRE, human
SRE can apply these fixes.
But, I mean, we have the permission model where we can give some of the,
this agent to automatically apply these fixes, themself, rather waiting to SRE.
We can say, "Okay, is it only config change? I will allow my SRE agent to
apply that change and fix the issue." So eventually it can take the more,
more like activity, but it's again, it's had the human control, in the loop.
So that's how the, we have designed it.
Like you said, the main i- intention behind it, so we running a system
agent, you can, you say, survey agent.
We know what are the context we need, feed into this agent to
provide a better, better result.
So that's how these inbuilt agents are, acting.
Sameera: Yeah, and I also think, like, if you're running something,
this is, these agents are a good starting point for you to come up
with your own agents as well using.
And then you can replicate with, you can, come up with your own agents because the
context problem is kinda solved i- in, within the platform, what Lakmal said.
So you can… Because these are all about not exposing Kubernetes details
into the agent, but exposing the OpenChoreo abstraction layer, so that
way it's what we have figured out.
I mean, not sure whether it'll work.
Bret: that's the hard part.
Yeah.
Sameera: Yeah.
Figure out is this abstraction field helped us to give the right context also
Bret: Yeah.
all right.
so I was writing this article over the last week around what does an
AI Kubernetes platform even mean?
And what we're really talking about here is something that I had to define
in this newsletter as what I'm calling type three AI on Kubernetes . It sounds
really nerdy, but if you've been going to KubeCons since the invention of
ChatGPT, for most of the history of what we now call gen AI or whatever-- we
are not talking about AIOps and MLOps.
I mean, technically you could use this thing to manage that too, but for those of
us that go to KubeCon, there's been-- I've been ranting a little bit for a couple
of years now that there's been this weird disconnect where when a lot of people
in the industry talk about Kubernetes and AI, they're actually talking about
running AI or making AI, building models, reinforcement, learning, running
inference, and that, that is not my world.
Like, I am not that person, right?
It's interesting from an architecture and engineering perspective, but
I don't think I'll ever… that to me, that's a speci-specialty and it
might get easier to do, and some of us might have to have that new role.
But I feel like there's still a significant, maybe even majority
of us that won't be doing that.
our new job is to build out these agents and understand where the, you
know, the features and the edges of models can be implemented so that
we can automate and build more.
Because I see this whole thing as just like VMs, the cloud, the invention
of the PC, you… I-over my 30-year career, we've had major pivoting points
in technology that essentially allowed us to scale ourselves as a human.
We went from managing 10 servers in the '90s to 100 servers with VMs to
a, a thousand servers with the cloud.
Maybe containers allowed us to run 10,000 pods per, per admin, and
then now this is the next level.
And to me, what you all are building is exactly that layer of abstraction, which
is the agents and their context management and permission management essentially.
that layer is now this new level of abstraction that will
hide some of the complexity.
Like, we don't have to know every kubectl command in the world anymore.
In fact, I don't even know how we all get Kubernetes certified because if
we're not gonna be running kubectl and have to know every option, what
is a Kubernetes admin test other than just understanding architecture?
But my rant here is that like I had to go through… This post took me a re-week
to write because I was trying to theorize around we're really talking about agentic
operations, and that's where I'm-- that's the reason I have the new podcast.
That's the reason this project is exciting, is this is that new layer.
This is the abstraction that we're all kind of searching for
because I think we've understood the local harness now after having
those for a little over a year.
I think a lot of us are like, "Yeah, I, I get this." We have all sorts of neat
ways of managing these little local agents, but this new nebulous layer that's
sitting in front of our infrastructure is still not fully realized.
So I feel like you all are pretty early in the game in terms of saying, we can help.
We have this defined component, and we can shove it in there." And, I'm even trying
to draw out a diagram of what- If we're all on a maturity path to, we started
with the agent harness locally, and the end goal is we've maximized the agent
assistance that we're all gonna have.
Like the dozen agents we're gonna have, two dozen agents, we've maximized
all the functions and features of what models can do for us as DevOps
platform engineers, SREs, right?
all of us are managing this infrastructure and there's a dozen tools and f- the
thousand features we all now need to learn, and we're learning skills
and agent files and MCPs and where all that can go awry and go crazy.
But eventually the idea is we're improving our productivity, we're improving our
management, we're hopefully making things more secure and reducing,
outages and everything's getting better.
But at the end of it, what are we doing?
And I think a year ago we were all kind of nervous like, are our jobs going away?
that's been a big conversation.
are we-- Is there only gonna be one DevOps person in the company?
But to me, this is the job, like this is the new job, is you all built
this, now we've got to implement it.
Now we've got to understand it, we got to understand the components.
Where do the agents help?
where do they not help?
where do we still need the human in the loop, like you said?
and I just wondered if that's, if any of that is anything that you would
agree with or that I'm… Am I crazy?
Does this all sound like a good idea to talk about?
Lakmal: No, I think, this is great, Bret.
I think you're, you are correct.
So yeah, it's evolved last one year, within the one year, we can run agent in
production, like helping to the different personas engaging within the platform.
I believe the generating code problem is almost solved now with the Claude
Code or other Cursor or Codex, right?
So people can generate code, write application within minutes So we
call it as vibe coding, right?
Everyone call it as vibe coding.
When the moment they hit the deployment, their vibe is ending.
Because now with vibe coded, application has to be promoted into the production.
But still, the platform doesn't support-- they have to m- go away from their agent.
They have to either create a ticket or like, go to the self-service
portal, and deploy the application.
But what we have seen, within the, our, skills and MCP servers,
they can use the same, agent they are using to develop, their code.
They can say, "Okay," I can say, "I want to deploy, the, my application
into the development environment.
Just you figure it out how to do it." So now it's, we call it vibe deployment.
So OpenChoreo try to fix, fill the gap, vibe deployment part,
how the operations side of it.
So you can write, agent, you can write your code using agent.
Now you use, you can use the same agent to deploy your application into the
production and operation, with support of OpenChoreo, section as well as the
tools supporting to the MCP servers.
that's, where we are, we want to play, around this, platform
engineering side of it.
Sameera: Yeah.
And Bret, I think you mentioned, some-something about how
abstractions help agents, right?
I think the way I think about that is more abstractions, fewer tokens,
Bret: Hmm.
Sameera: so they don't have to learn Kubernetes deep YAMLs and kubectl.
like, I think the same applies to programming languages in a way,
writing assembly versus writing in Java or C# or Go, right?
fewer tokens in that way.
So they, they have more context with less tokens.
I think that is also, that's what I'm, working on these days, see
whether we can establish that.
Bret: Yeah, that's a, I think we're all very quickly, especially these last three
months when it comes to tokens, we're very quickly understanding that this is finite,
and that, just kind of like Kubernetes itself, we've all had that moment where,
early in our Kubernetes journey, we would realize how much infrastructure
we actually need to just run the apps.
Like we, you know, the, when you build out a full-scale Kubernetes
cluster, th- there's a s- there's a non-insignificant amount of infrastructure
to manage the infrastructure.
And that, back when we first started the journey on Kubernetes, I don't
think we all really understood, the eventuality of dozens of different
controllers and dedicated nodes in the control plane that we were gonna have
to have just to organize and Uh, to, to herd the cats, as, they might say.
And, I feel like now the same thing is with tokens.
It's like we got all excited, and we're like, "We're gonna put it everywhere."
And now we're gonna very quickly end up in a world where we've got budgets,
and now we've got to optimize our agents, and we have to care about the
model, and we have to, we have to…
One of the conversations we're having in the Agentic DevOps Guild is around
evals and how can we use evals, to sort of figure out which model, if
I can apply a bunch of skills, if my agents are really just behaving with a
bunch of skills and context management stuff, how can I, sort of evaluate or
programmatically determine or test that maybe I could use Kimi or, MiniMax or
Qwen or some, sort of open weight model that's super cheap for my very specific
little, you know, FinOps niche agent.
Whereas it maybe doesn't need Opus because maybe that's the one that
I need because I have dumb prompts and I s- I type dumb things.
And maybe my agent in the cluster can be running really cheap because none
of us wanna put these in production and then find out a month later that it's
too expensive to run, and now I have to shut it down and make the humans
do it again because I don't think that's a world I, I wanna go back to.
Lakmal: That's true
Bret: I know a couple of companies that are playing in this space.
We've had Anyshift on the show, we've had, Mendral on the show,
and these are both AI startups that are building agents to help with
infrastructure, CI/CD, sort of helping, helping you with fault remediation.
and the pro- approach that I've been seeing everybody take is,
like, it's just sort of the normal pattern of engineering approach.
We walk before we run, you know, we r- we run before we sprint, and that means
on day one, you might be read-only.
agents can't do things, they can just go look up things, and that
they're just a little helper assistant, and they're not gonna…
They have no control over anything.
They're just here to help you find information in the sea of,
of infinite information that we have about our infrastructure.
And then, then, like, with Mendral, the team with Mendral that's been on
the show, they're now to the point where, very specific use cases, they're
letting the agent have a tiny bit of control because it's predictable.
They can test and make sure that it's, that it works, that it-
it's as declarative as possible so that it is predictable, and
they start to, to let that happen.
is that something we can do with OpenChoreo?
Like, does this have sort of a default stance of, "I'm only gonna be read-only"?
how does, how do the permissions work that way?
Lakmal: Yeah, it's not only the read-only at the moment, but you can control it.
we have the permission model where, you can give the, different
tool to the UI, your agents.
It can be read-only tools, it can be write operational tools as well.
Based on the, what we want to do, you can allow this permission, because agent
have their own identity, and they, within the identity they have the permission.
So based on the permission, they can use different tools.
if you are confident enough that your agent is acting, performing well, you
can give the right permission to some s- some level of factory, for example.
you can give like, okay, if there are configuration issue, agent can go and
fix it itself, the configuration, fixes.
So you can fine-tune how you, your agent can interact with your platform
Bret: Yeah.
I've been wondering about a lot of this around, like, how is per-
how are permissions gonna work?
so much of our permissions models today feel like they're not designed for an
agentic world, and, how are we gonna track the actions of these things?
Like, every little step they take, is hopefully gonna be put into some
sort of observability platform.
So I'm, I am very curious to see how, what your all's vision is.
Okay.
Yeah, I'm trying to envision, what ends up… Like, how to correlate this to
the things we know today, like GitOps repos with Argo CD definition files
in them that, that get, you know, pulled in because Argo's watching
or Flux is watching a different repo and it's looking for YAML changes.
And, I'm assuming, does this change that workflow?
Does… I remember seeing GitOps on the diagram, so I'm kinda thinking,
like, how does this correlate to sort of that, that GitOps loop that
we've seen so many teams adopt?
Sameera: Yeah, I think these, the CRDs, right?
so they, you can put all of these in your, in your GitHub repo, and then you
can configure Argo CD, configure Flux.
the normal workflow will work as it is, right?
Bret: Okay
Sameera: And so there are two modes, right?
Here, I think we are using more ClickOps, ClickOps approach, but you can configure
the same thing with, your GitOps.
Bret: Oh,
Sameera: will talk to… Yeah
Bret: I see.
S- so yeah, so in this case, what we're saying is like, I mean, yeah,
we're looking at a web dashboard, but we're not really… We're j-
I mean, for the audio listeners, we're, we're looking at a dashboard.
We're not really, we're vibe opsing, I guess.
Not ClickOpsing, 'cause we're not clicking on the
Lakmal: Yeah
Bret: dashboard.
we're using the agent, but the agent is causing… Just to be
clear, this is actually a question.
So the agent is causing OpenChoreo to create new resources inside the
cluster for deploying this inf- this, the Google, microservice demo.
And I, that is an alternative to having an agent that I just say, "Hey, we're gonna
make a new PR in YAML, and you're gonna push that to GitHub, and then Argo or Flux
is gonna pick that up later." because I guess I could, I could tell these agents,
or I could give them skills or change the skills somehow so that it knows that
that's my workflow, and that I'm just using it as sort of a read-only partner.
But kind of just like we're not supposed to go into the AWS console
and ClickOps away there, like, I don't necessarily want OpenChoreo changing my
infrastructure outside of my GitOps loop.
I'm actually just curious, do you think that the GitOps approach is less
important if we have this sort of chain of events and… or do you, like, do
you see enterprise teams possibly mo- shifting to more of this vibe approach
since we have something in the middle?
Or do you think that, you think that GitOps is, like, here to
stay and that's still the s- sort of more mature, safe approach?
Lakmal: Yeah.
I would say, when you come to the lower environment, this a-agentic
directly creating this re-custom resources will play a big role.
because, but maybe in, in production, when you promote in the production, people will
use, GitOps or declarative way of defining these things, in-into the production.
but eventually if the human will confident enough with your agents
a-and, trust your, agent, eventually I would say, people agent can directly
call to the MCPs and create the, like, the create custom resources
directly within the class itself.
Bret: Yeah
Sameera: way I think, exactly.
Lakmal: in the this year, but maybe in future.
Sameera: I mean, one, one adv- the way I think about that is, I
don't know, one advantage of GitOps is you have the tracing, right?
you know what, what happened exactly over the years.
Probably, yeah, dev environment probably you don't need GitOps,
like you can give developers full capability, do whatever you want.
But for other environments, I don't know, my view, like you would be… I would
still use GitOps to control the workflow.
Yeah
Bret: Yeah, I've started to wonder if, if the agents will eventually sort of
like- We're interacting with the local agent on our Claude Code, and we're using
these skills and, I think everyone's already writing the YAML with agents.
Like, we're already at the point where teams that have u- are using
Claude Code, they're not handwriting Kubernetes YAML anymore, right?
they have the agent do that.
Why would, why would I wanna do that anymore?
And, I wondered if there was m- 'cause there is a mode in Argo where, like,
you can sorta do things retroactively, where maybe you make changes,
but then you document them after you're, after you're, you're done.
And I wondered if agents possibly created a future where GitOps was more re- m-maybe
not the first thing we did, but maybe more of a, a system of record, but done
after and if agents sort of were pushing us into a faster evolution of this.
a constant conversation we're all having is around sandboxing and,
specifically, our local agents.
one of the challenges is, I'm just doing stuff on my local machine all
day, and I don't necessarily… And every AI that I open up by default, even
if I use, like, Claude Code built-in sandboxing, chances are it has access
to my AWS, my, my Docker CLI, my kubectl CLI, my Terraform, all these things,
and those keys are all already authed, and it could just deploy to things.
And so one of the, one of the things that I'm trying to work on with
some friends is, where could we draw a boundary around sandboxes?
And this almost feels like a very perfect scenario of these MCP tools and
these skills are maybe all in a Docker sandbox, which is more of like a VM with
a, with the harness running inside it.
And I only use that particular harness because it has these particular,
permissions, and I only spin that one up when I want to… It's almost
like per environment, configuration, and each Docker sandbox tends
to have its own configuration.
the, it's… The way at least it works now is it's very, things are very isolated,
including all your keys and everything.
It, it doesn't really bring a lot into it, so you're sort of isolating
everything into different harnesses.
So I'd have, like, my prod harness, my, my staging harness, and I can interact
with those environments, and keep them safe away from, like, my normal day-to-day
harness that kind of has maybe too many… a lot of small teams, too many
keys are on their local machines, right?
they know it.
They're out there.
if you're a team of three, chances are you probably got the production Terraform key
on your machine, and you're just like, you know not to type the Terraform commands.
you know that's the human thing.
But now that we have the agents, I think people are getting, starting
to get a little more concerned.
They're like: What if it accidentally picks the wrong environment?
Or if it-- I accidentally have the wrong key and it just starts going?
so this feels like a very, very interesting approach of Each environment,
or, I'm assuming this thing manages, can manage multiple clusters in one
OpenChoreo, or is it a per cluster thing?
Lakmal: It's one multiple clusters you can manage within OpenChoreo.
Bret: Yeah.
so yeah, so I, I almost treat it like an environmental gateway t- for my agents
where I'm like, I call this production and I have this special Docker sandbox or what
other sandbox I wanna apply, and the keys to production, even if it's read-only,
are only accessible from that thing.
And maybe also I'm gonna have some GitHub keys in there so that my agents i- in that
sandbox can also write to the GitOps repo.
Maybe they can make the PRs for me.
but I'm now realizing how these things are coming together, where I can have
the one local agent that can see the infrastructure through OpenChoreo,
but can also implement GitOps changes based on the information that it's
getting out of the infrastructure.
So if the pods and, you know, a pull back off loop, I can see that
the agent can determine that, write me the GitOps in the same session.
It writes the GitOps b- diff.
We're pushing that to a PR, and that's how I'm gonna be iterating on my cluster now.
Or maybe I'm doing it in Slack while I'm at lunch, and it's all just through a
Slack bot, and I'm not even… I'll ask you these questions to make sure that we
have them, have them at top of mind here.
"I like it's Claude Code that can interpret or understand,
promote into staging. Can we use a private LLM to interact with
OpenChoreo instead of Claude Code?"
Lakmal: yeah, so it's, it's totally, in Claude Code, it's,
it's your, your model, right?
So it's a, Anthropic model.
You can use any Codex or any other agent.
It's just MCP servers you are interacting with.
You can use any LLM, in your local agent.
And also in the inbuilt agent also, you can configure your LLM.
you can either OpenAI or whatever model you can configure it.
it's a configuration, for us
Bret: Right.
Yeah, picking the model for the agents, is, is a good thing 'cause we're gonna,
we're, we're-- I got a feeling we're gonna need to get cheaper models for the, for
the cluster, for the always-on activities.
yeah.
and then, does the agent have visibility into inter-cell dependency traffic, or
is it context window completely blinded to anything outside its assigned project?
Lakmal: i-it's depending on the different agent interact with, different contexts.
So for example, say, in this case, SRE agent, when that troubleshooting, it
can, go beyond the one project, because some component will interact with the
other component within other project.
So, so if there are interaction, this agent can look at, what
happening in the other, other component, in the other project.
So it's b-based on the context that agent are interacting with, they can go beyond
the single project or multiple project
Bret: Right.
Okay.
And just to be clear on, the scope of this thing, it doesn't appear like you're
trying to boil the ocean here, where OpenChoreo is the single control plane
for AWS, for GCP, for all the other things that we might have to manage.
it, it's trying to remain Kubernetes focused, right?
So, like, the lowest layer in our infrastructure stack is, I guess m-
maybe it could possibly get some of the technically OS level stuff underneath
Kubernetes, but that's, that feels like that's the s- the dropping off point,
'cause I don't see, clouds listed in here.
Lakmal: Yeah.
we have, we are running on top of Kubernetes.
We are not even managing the Kubernetes with OpenChoreo.
we, or we build on top of the Kubernetes, so you can run
OpenChoreo on E- EKS or EKS, but we are not managing the EKS or EKS.
So that how we architect.
But, OpenChoreo has this resource abstraction where you can manage the
cloud resources, within the OpenChoreo.
So we have integrated with, Crossplane at the moment.
So OpenChoreo control plane can in- talk with the Crossplane integration
and via the Crossplane, it can create a, say, S3 bucket in AWS, and it can,
do the lifecycle manager for S3 bucket.
So it provide a single, unified control plane to manage it, but,
it's up to the users if they want to, use, manage in different way.
That's totally up to the users
Sameera: Yeah.
Just to add to that and then, Bret you saw like multiple planes, right?
Control plane, data plane, observability plane.
And you can run each plane in its own Kubernetes cluster, or you
can run all planes in one cluster.
And you can run your control plane locally, you can run data plane in AWS.
So it's, as Lakmal said, it's, the way it's architected is so it's,
you can configure, the way you want.
And then like, like for example, if the data plane is in AWS and control plane is
somewhere else, you don't have to expose the Kubernetes API server to the internet.
There, there's a certain agent running in the, in the data plane that creates an
outbound connection to the control plane.
that's sort of the standard these days
Bret: Right.
So yeah, basically, if you're gonna use this to help manage something, it needs
to be something running in Kubernetes, and it needs to be probably like a s-
something around the CNCF ecosystem of tooling or something like that.
So this isn't trying to replace Terraform or, you know, control my
cloud formation, and stuff like that.
Yeah, yeah, or run my, my, my other infrastructure like
Vercel or something like that.
Yeah.
that gives me a nice boundary because like when we talk about the future of
what it's m- means to be an SRE, DevOps platform engineer, whatever, like where
do the edges of these tools exist?
Because, sure, I can manage everything from Claude Code in theory at this point,
but, it probably, I probably need very scoped things in order to keep it in line.
And having this thing w- well defined is like it runs on top of
Kubernetes, so things that you can do in Kubernetes, it can help with.
But don't-- you maybe can shoehorn in a bunch of things like, for AWS or
whatever, but maybe it's not the right tool to do everything all in one place.
'Cause obviously there's a ton of context outside of Kubernetes that
it would probably need and, yeah.
So there's probably other tools for that.
At the end of the day, we're gonna have dozens of agents, I feel like, and we're
just gonna, we're gonna have an agent management plane where we're all just
staring at agent configs and skill files.
And that's like, that's our new job.
More markdown.
So how can people get started?
They go to openChoreo.dev and or is this a Helm chart?
Like how do we get started with implementing this?
We, well, step one, you gotta have Kubernetes some- running somewhere, right?
Sameera: So yeah, if you, if you go to OpenChoreo dev, documentation
page, there are m- couple of ways for you to get started.
First thing, if you wanna just try out OpenChoreo, so we have this quick
start guide that'll basically… The quick start guide that'll basically…
You just need Docker for that.
So Docker and Kubernetes, we will do the, we will…
There's one command that'll install Kubernetes in Docker, and
then that'll install OpenChoreo.
So you get a… Once it is done, you get a UI, you get MCP access, all that.
That's just 10 minutes.
won't, pollute your local environment in that way.
You can just destroy it after that is done.
The other option is you need to have Kubernetes cluster on your
local machine, install OpenChoreo.
There are, then the third option is install OpenChoreo in cloud environment.
That's so, that's sort of the way how we have structured getting started.
Bret: Okay, so it's got some persistence to it.
I'm assuming this thing, does this thing have, like, databases
as a part of its deployment?
Does it have, like, a Redis or, like, what-- When we talk about the
infrastructure inside of OpenChoreo, I mean, we've got a web portal, we've
got some, obviously some MCP endpoints, we've got some system components based
on other things you're installing.
what's that persistence layer look like for us that are gonna have
to deploy this on our clusters?
Sameera: for the, I think for the whole OpenChoreo control plan, I would say
right now the main persistent layer is ETCD, whatever the Kubernetes use.
For Backstage, you can configure, Postgres or any other database
that Backstage, recommends.
Right now it uses… Yeah.
Yeah.
I think apart from that, there's no persistent,
Lakmal: Also, for a, like a, the SRE agent, you can configure, it's
coming with SQLite, for the saving the incident and providing the
context in the future incidents.
But you can configure a Post- a Postgres or any other database.
Bret: Yeah.
I was actually wondering too, I was trying to imagine, this thing is doing
things, these agents are doing things, and we probably want a history of that.
Is this thing able to, log into your existing… Like, is it logging
through Kubernetes to your existing… So it's essentially creating more
m- monitoring and logging data in your infrastructure by the act of it
just being there and doing things.
Is that, is that where I would go to see, hey, what does my
agent do in the last 24 hours?
Is it something like I would just look at my normal monitoring
and logging platform for that?
Lakmal: Yes.
it will do all the audit logs, to the, log itself like we are using, depending on the
lo- log module, it can be, OpenSearch with a different index with, for the, audit
logs, what agent do, whether human do.
Everything is tracked, within the logs itself
Bret: Okay.
and all the data, about the infrastructure, are the agents
just pulling that real time?
Or do we, do you have to, deal with caching, you know, infrastruc-
infrastructure graphs or any… Do you have to do anything like that and
to optimize in the OpenChoreo layer?
Lakmal: Yeah.
At the moment, it's a real time.
we are not using any, VectorDB or something like that, for the
caching or something like that.
At the moment, it's, real time pulling the data, but we give the
time interval, the context saying, "Okay, within this time interval, you
have to pull the logs, matrices." it will just look in that interval and,
get the, all the data to troubleshoot
Bret: Right.
So it's, so it, once it sees an event, it already knows exactly the second
that event happened, so then it can limit itself to th- so that you're not
burning a bunch of tokens by boiling the ocean and scanning every log on
Lakmal: Yeah.
Bret: infinity.
Okay.
Lakmal: the idea
Bret: Yeah.
Perfect.
is it, does it need to look at, like, Grafana graphs?
Like, or is it just simply doing prom queries and just getting
that data real time as well?
Lakmal: It's the data to the, observability MCP server.
It's, just going to the MC- MCP servers
Bret: Yeah.
RIP Grafana.
Sameera: Yeah.
Bret: Yeah.
Oh, sorry, what would you say?
I was talking over top of you
Sameera: No, this is directly from, observability MPV service talk to
the OpenSearch and get the data.
Bret: Yeah.
Yeah, yeah.
Yeah, so that it's a little bit simpler than it having to
constantly, figure out everything and store it as some sort of cache.
Which, I mean, in theory, that would make the agent faster so that it didn't
have to query everything itself when you start to work with it, but it also adds
a lot of complexity and, and resource utilization, which gets back to the
point of at some point, everybody has had that infrastructure where they
realize they, they have a, a cluster that has more resources used for the
infrastructure than the actual apps themselves, which, you know, has happened.
sometimes when we have small clusters and we're getting started, like we are
like, "Yeah, you know, the, the app only costs $100 a month to run, but
the infrastructure costs 10,000 a day."
Lakmal: That's true.
Bret: yeah.
Well, this has been very cool.
we could honestly, I could talk to you for another hour about this because
I'm really, I'm really interested in the patterns and the, and the ar- the
architecture design that you've learned, because I think these, these are things
that we're gonna have to do the same thing for our cloud infrastructure.
We're gonna have to do the same thing for our line of business apps.
We're gonna have to s- figure out how to give these agents under- a deeper
understanding of that infrastructure so that we can depend on them more and
rely less on the human, the human tribal knowledge that we all have about how
these systems are put together and where all the bodies are buried and where all
the, My, the friends, my friends of mine, they call it the sins of the data center.
Where are the, where are the sins of the data center at?
And this thing is gonna have to know all that.
this thing is really, when I say that, I mean like dozens of agents that all
have certain context and certain control.
but it all kind of feels like it's coming back to this still, this, Viktor Farcic,
who runs the DevOps Toolkit channel, a friend of mine, he talks about that,
like to me, to him, Claude Code is the center of the universe and he manages
everything through it, and that anybody who's creating a project that isn't
expected to be local harness first as my interface is creating an outdated project.
So this kind of feels like it's, this, is doing exactly what his, he was
predicting over the last year, which is, you know, the harness is my gateway to
all this infrastructure, and everything else needs to be there to help me
be like local agent harness first.
and it's pretty cool to see that, to see that take place.
all right, so people get started at the openChoreo.dev website, and
then I think you're on, LinkedIn.
I'm trying to think what the project is on LinkedIn and X.
Lakmal: Yeah.
We have the X and LinkedIn pages, and also we have the Slack
channel, CNCF Slack channel.
we were actively, engaging with there
Bret: Yes,
Sameera: And we use GitHub discussions a lot.
If you want to engage with us, we'll be there.
Bret: I love to, I love GitHub discussions.
I d- wish they were used more, r- rather than issues.
Like, I feel like a lot of things just need to start in discussions before they
go to issues, and maybe some projects do that now, where they not, they don't
even let you post things in issues.
You have to, you have to start the discussions first.
yay, AI ruining open source for us.
at least for now.
We're figuring it out.
thank you both so much for being here.
you can follow the project, like s- we said, on LinkedIn or on X or, join the
Slack and the… Is it the, is it the Kubernetes Slack or the CNCF Slack?
Lakmal: CSCF lab
Bret: CNCF
Sameera: Slack OpenChoreo channel, I would say.
Bret: Yeah.
Okay.
I'm in both, and I can never remember which projects are in which and,
or if the Kubernetes… Yeah.
So it's in the CNCF Slack.
You can, probably just Google that and find that.
tell your agent to sign you up.
Like, don't, you know, don't go do it yourself.
thank you so much, Lakmal and Sameera, for being here, I'm looking forward
to seeing what's next in the project.
last quick thing.
What's next?
What's, what big PRs are coming down the pike since we're talking open source here?
Lakmal: More agents?
Bret: More agents.
Could-- I could have guessed it.
I could have guessed it.
Sameera: All about AI.
Bret: Yeah.
More, more of the AI overlords.
All right.
Thank you so much for being here.
thanks for everybody in chat for being with us, and we'll
see you in the next one.
Bye, everybody.
Thanks so much for watching.
I'll see you in the next one.