So essentially, go and assign that entire end to end role.
that can also perform in high velocity, high accuracy and high predictability.
And at the day, be fully audited for compliance reasons.
And of course, that frees up the humans and that
frees up your team's time to go and to innovate.
Welcome to Screaming in the Cloud.
I'm Corey Quinn.
Unless you've been hiding under a rock somewhere, you've
probably heard a fair bit about GenAI lately and how it is
the savior slash doom slash hype cycle to beat them all.
My guest today is Amit Yelgovrin.
Who is the CEO at Kubiya.
First, thank you for joining me.
I appreciate your taking the time.
Thank you for having me here, Corey.
Complicated environments lead to limited insight.
This means many businesses are flying blind instead of using their
observability data to make decisions, and engineering teams struggle
to remediate quickly as they sift through piles of unnecessary data.
That’s why Chronosphere is on a mission to help you take
back control with end-to-end visibility and centralized
governance to choose and harness the most useful data.
See why Chronosphere was named a leader in the 2024 Gartner
Magic Quadrant for Observability Platforms at chronosphere.io.
So you have been, uh, obviously a little bit on one side of
the GenAI, is it great, is it terrible, divide, given that you
have a company that is selling something directly in the space.
Take that head on, what are you building?
What I'm not building is an AI solution.
I'm building an outcome.
And the outcome is actually, maybe if we take one step back, we can talk about
what I'm trying to solve for, and kind of the paradox I'm trying to shatter.
And talk about where GenAI can enable that.
That's a good way of approaching it.
So often it feels like people are raising giant rounds just
because, Alright, I basically wrote a Python script that, Step
one, import OpenAI, and step two is, Meh, we'll figure it out.
Then they're shocked, simply shocked, When a feature
enhancement OpenAI puts out destroys their company.
Who would have predicted it could speak PDF one day?
And yet, here we are.
So what is the problem you're aiming at and what outcome are you going for?
That's a fair statement.
By the way, I'll just acknowledge that.
Let's be honest.
GenAI, most people discovered it under a rock about two years ago
when OpenAI officially announced themselves to the world as chatGPT.
Clearly, we've been doing this a little bit longer.
Uh, we're doing, we're, we're working with all the, you know, up to date models,
uh, training our own models, doing all the things that, uh, you know, you would
expect an AI company to do, but that's not the topic of today's discussion.
Clearly, that's that's for people to geek out with afterwards if they want docs.
Um, what we're actually looking to solve, because
that's, that's really why people are listening.
There's a concept called the time to automation paradigm.
Are you familiar with it, Corey?
I think it's better if you explain it to everyone,
because even if I am, I guarantee you, someone is not.
Actually, I want you to do the selling for me, so
Okay, on some level, if you wind up with It's a question
of, is the juice worth the squeeze, longer term?
The idea of how long do you spend automating a
thing versus how many times do you do the thing?
If you're spending three days to automate something,
you do once a quarter, that takes five minutes to do.
Cool.
Is that worth it?
Well, the answer is, of course, it depends.
But that's a loose definition of it.
You probably have a better one.
Actually, that's a perfect, uh, layperson definition of it, but it's really
the outcome and the effort that, you know, is the effort in time to automation.
The amount of time it takes to write the script, the Terraform
landing zone, the configuration file, and obviously maintaining
that golden path use case versus the number of times it gets used
or essentially the output you receive is congruent to one another.
And oftentimes you'll find that it's not.
Oftentimes a level of effort and determination and obviously the ongoing
maintenance it takes to automate an end to end process and the output that
you receive isn't necessarily going to go and relate to business outcomes.
And that's typically where you see a lot of organizations
with a very clearly defined automation strategy.
Coming in with, we're going to set up an internal developer
platform, uh, name it backstage or any other deviation of that.
We're going to go and set up some kind of, uh, self service platform.
They're going to go spend all this effort, oftentimes
even the headcount associated with it, just to find
out that after a year worth of, uh, A lot of toil.
They managed to set up seven golden path use cases out of which the
first time a developer tries to go into self serve themselves into
one of these automations, encounter some kind of configuration or
access or permission issue, goes right back into the GR ticket queue.
Back to the on call engineer and says, Guess what, buddy?
Enough is enough.
Enough of this nonsense.
I need a human to help.
And that's typically, and it repeats itself in various different formats
and different flavors and organizations, but that's the sum of it.
The paradox, just like Joubin's paradox in many respects, if it takes longer
to automate something than it is the output, it's not going to get done.
And what you find is many organizations start with the strategy, end up
going down the line just to find out everyone's doing ad hoc scripting.
Until what happens
You do tend to cut yourself to ribbons on the edge cases though.
Okay, this only takes five minutes every
morning to do, why spend a week automating it?
Well, it's done every morning, sure, but look
at the defect rate when humans are doing this.
Sometimes people are hungover, sometimes someone is out sick.
If you can, If you can get a better outcome via automation,
that does tend to put a thumb on the scale from time to time.
But yeah, directionally, you're spot on.
No argument there.
And I think where kind of the output and kind of the outcome base we're
referring to, it all comes down to The amount of effort versus the output.
If it was as easy to automate an end to end process
as it is to have a conversation with, say, Bob.
And Bob's your on call engineer.
And every time you need something, uh, from a platform, you just
go to Bob and say, Hey, Bob, can you go ahead and configure this?
Uh, this resource for me.
Can you go ahead and grant me access or permission to this
IAM policy or creative policy, elevated permission, or get
approval for this resource that requires that approval for this?
If you could do this as easy as having a conversation with Bob.
Guess what, Corey?
It's going to get done every single time.
So what we've created, Kubiya, and this is why you mentioned GenAI
at first, and you kind of set this up for this answer, is if instead
of Bob, you have an AI teammate called Kubiya, okay, or Kubi Jr.
And in this case, you could have a full on discussion,
conversation, bi directional conversation, and it is access
aware or mission aware and able to really meet the users.
In the exact channels where they communicate and
collaborate already, Slack, Teams, Jira, Kanban boards.
Then you could have that personalized experience and release
Bob to actually do the real work, which is setting up the
infrastructure to tee up the company to have the next GenAI company,
uh, product that they're going to go and roll out themselves.
Really, it's all about outcome and rewards.
If you're going to want to go and put all this effort, it better
be worth the reward, unless it's as easy as having a conversation.
Then you break away from that.
The counterpoint, of course, is this has been teased at, and people
have done a number of experiments, myself included, of, all right,
I want a Python script to do X, go ahead and build that out for me.
And ChatGPT gets some parts right, some parts wrong, but it is
very far away from being something that I could accept out the
gate as meeting the acceptance criteria that I have for it.
It, it feels like on some level, I like to dive right back into
your paradox that I would spend more time supervising the thing
than just writing the quick script myself in some of those cases.
How do you get around that?
So you hit the most important point.
It's not just about doing the automation, because that's half the battle.
It's about doing it in a way that's expected.
controllable and fully auditable.
And we're actually allowing that.
We're actually using Terraform as our backend, in many respects, to give
the user the ability to control every aspect of this interaction with, uh,
with, uh, essentially, uh, the teammate that's configured with Terraform.
So you get to control the environment variables, you get
to put the output, you get to control the permissions,
and you get to control every single aspect of it.
So as an operator, you're in full control.
As an end user, you're interacting and having that LLM type
of experience that people are accustomed to with chat GPT
and other type of chatbots that they're comfortable with.
So you're getting the best of both worlds.
I want to dig in a bit on the idea of talking about this as a virtual teammate.
Um, specifically from the perspective of, I don't know about you,
but I have something of a potty mouth when I'm berating Siri when
he gets something wrong or Alexa when, basically, I ask for anything
and then I'm followed up with a, By the way, buy some more pants!
or whatever it is they're trying to sell this week.
Um, if I talk to an actual colleague like that I, HR is inviting
me to a meeting in which I'm not offered coffee and very shortly
afterwards, I'm not allowed back in the office ever again.
Uh, so there's a, there, there's a question of how much is this a
accelerational tool for folks that need, that are, that are getting
value from it versus how much is this actually intended to be a
full on, uh, member, the fourth person on a three person dev team?
Like, is this a, is this a, I guess employee replacer isn't an augment.
Where on that spectrum do you see it landing?
So just like you would say across all sorts of revolutions,
industrial revolutions, where the humans weren't replaced by
the smart assembly line, they became supervisors of the smart
assembly line and managed to go and to reinvent their position.
And you could go down the list from the cloud and how you did things on
prem to the cloud and how you went through all the different resolutions.
At the end of the day, AI is a tool.
It's an enabler.
It's a megaphone.
If your entire role in an organization is to move a pencil
from right to left, then likely AI will replace you.
I'm sorry to say that, but if you actually have the
capability of supervising and becoming an enabler of
essentially a supervisor of agents, so think about this as.
Up until now, you've been an individual contributor.
Now you actually get to supervise your teammates.
So you're a DevOps manager all of a sudden.
That becomes a completely different job title.
And of course you get to see everything through
and have the highest and best use of your time.
Freed up for the things that AI aren't prepared to do.
And some of you are talking about moving
commoditization a bit further up the stack.
Uh, similarly to, used to be you had to run a bunch of compiler
commands to get a web server, then to run your application, then it
was just Yammer app installed, then in time it became, oh, now it's
Docker run and you get the whole thing prepackaged, ready to go.
And you're spending more of your time, Trying to get the application to do
what you want it to do and not get the application set up in the first place.
That's exactly what we're saying.
You don't want to have to do the repetitive work
that otherwise would have been better suited for AI.
Free up your plate.
You have plenty of work to do.
You probably have a big backlog that you haven't even gotten around to
because you're behind the eight ball every single day when you start.
There's a hair of fire drills.
Do more with less is the persistent rallying cry of
our current industry and honestly our entire system.
There's case studies that are being studied
in Harvard and every single business school.
It's all about Blockbuster and Netflix, right?
Don't be Blockbuster.
Don't be left behind.
If you know how to reinvent yourself and adjust yourself,
AI is going to be the biggest enabler, biggest career
boost you could ever have in your, in your career.
Otherwise, if you feel that you're perfectly fine with stacking DVDs
and dropping them in the inbox every single Sunday when people have
to return them, You're going to be left behind in the streaming,
uh, in the streaming kind of movement will take you by storm.
That's kind of what we're saying.
Don't be left behind by AI.
Be, have AI be the enabler for you in your careers.
I don't necessarily disagree with the premise.
I think that it is It is fairly clear at this point to most
folk that there is value that can be derived from GenAI, whether
it is this wild transformation of society to a perfect utopia.
I'm a little bit of a skeptic, but it's, it's similar to,
oh, I, I insist on doing long division the old way because
I'm not a fan of these newfangled things called calculators.
Yeah.
It, it acts as a tool that accelerates, but understanding
when to apply it, how to validate the output that comes
out of it, and to inval to ensure that it's not insane.
is going to be something I think that we're stumbling through as a society.
And, and in many cases, the hallucination problems, uh,
aren't making a strong case for, let's turn the air traffic
control system over to the GenAI and hope for the best.
There's a, I think that it's a matter of nuance similar to before this,
People, developers would wind up using Stack Overflow, the world's premier
copy and paste website, and use that to solve problems on an iterative basis.
You can amalgamate that into various coding assistants and chatbots.
Having them actually go ahead and do the implementation
seems like the next logical step, but as always,
there's going to be some question around the margins.
And how this, is this going to be something that we can actually trust?
And if so, how far?
So the beauty of what we're trying to accomplish here, and it's not to let AI
take over the entire end to end workflow and orchestrate the entire process.
You can't avoid hallucinations.
By the way, that's a feature within large language models, right?
It's a statistical, statistical based approach.
Every single answer will deviate from the other answer.
Every single time.
What we're actually advocating for is to make it very controllable,
very predictable, and that's where the Terraform code comes into play.
And the AI enablement is essentially the free, the natural
language interaction that you have, where you can go
and abstract away the business logic of your intent.
And then work into that, uh, pre defined, pre gated workflow.
So it's essentially combining the best of both worlds,
both the known and expected structure, along with the, all
the things we know and love about large language models.
I think that that's a fascinating approach.
I mean, something that I've always done when I've been
asking large language models is I won't ask for the answer.
Because, okay, you're going to give me an answer.
Maybe it's right, maybe it's wrong, but regardless, you
certainly sound very confident in what it is that you're saying.
What I'll ask instead is for a script to go ahead
and do the thing to get the answer out of it.
Because that, from my perspective, that gives me two great paths.
One, I can see how it's doing that and potentially
catch weird issues it's making along the way.
And two, okay, that was great.
I want to iterate on that now.
I'm not going back to square one or trying to find the chat that generated
that and then have it, uh, have it go ahead and pick up where I left off.
I find that that, the show your work stage and breaking it down into
stages means that when it starts to go off the rails around step 17,
you can go back to 16 and try again to get things moving along again.
I think that that aligns with the approach that you're taking.
It's a very modular approach and the ability to
go and to insert your own tools, your own scripts.
your own code as part of this to inject that into the process.
Make sure that you control every aspect of your workflow.
It's essentially your own words orchestrating essentially an end to
end complex process that otherwise would have taken disparate tools
and processes within the organization to accomplish in the same way.
It's just done in a highly, highly condensed time
to automation, which is the beautiful part about it.
We can go into use cases if it helps.
By all means, give me an example use case.
I like, let's talk about something real, rather than
the ephemeral vision of the developer of tomorrow.
Let's talk about something that, uh, that someone might actually do.
One of our favorite, um, golden use cases, if you may, is,
uh, one of our customers came to us to Enable a self service,
um, infrastructure or resource provisioning platform.
All within Slack, which is obviously where they meet their users.
So, the concept is a user comes in, asks for,
I don't know, a new SQS queue, for example.
And this is, uh, as part of an application that they
want to copy over from one of their other resources.
So, the ability for the teammate to first verify the identity of
the user, verify that they have permission, maybe even create a
just in time, uh, policy in order to enable that user to do so,
but then also backtrace, uh, what the cost of this resource would
be, because there's budget enforcement that has to come into play.
So if it costs more, say, than a hundred dollars a day, that
requires some additional layers of approval, which again, the
teammate could also go and get the right approvals for that.
So the ability to go and to both enforce budget, enforce policy.
And create least privilege, uh, automation without needing to assign a role.
That's already a big win for this organization.
At the end of the day, they also care about cost.
So not only are you enforcing the budget, they also have a cleanup process
where after 30 days, you manage a TTL, you could actually configure that.
3 hours, 3 days, 30 days, it would go and automatically
destroy that resource and bring it back to where it was before.
So you're never, you know, over provisioning or over
resourcing, and it's all done as a simple conversation.
This same process, if you would have copied that over to the way they
currently do it, would take a matter of three to five days and five different
people involved in the process to provision and de provision that resource.
With us, it's less than a minute.
Complicated environments lead to limited insight.
This means many businesses are flying blind instead of using their
observability data to make decisions, and engineering teams struggle
to remediate quickly as they sift through piles of unnecessary data.
That’s why Chronosphere is on a mission to help you take
back control with end-to-end visibility and centralized
governance to choose and harness the most useful data.
See why Chronosphere was named a leader in the 2024 Gartner
Magic Quadrant for Observability Platforms at chronosphere.io.
I think that there's a, there's also significant value in being able to
spit these things out that then go through a somewhat normal production
process where, okay, great, this works in a test account, it can go ahead
and spin things up, whatever, ideally there are guardrails somewhere around
it to prevent it from doing the psychotic things that make the headlines,
but okay, once that's done, great, then having it be vetted as it gets
promoted to higher environments and have humans weighing in on that.
does seem like a reasonable control because objectively there's not
that big of a deal when you have a GenAI system hallucinating or
being wildly inappropriate unless you're deploying that directly
to customers without any form of human review along the way.
I think if you put a chat bot on your website and let it
and give it make it authorized to cut deals on your behalf
and then it does horrific things, I think you're unhinged.
I think if there's a human review that goes through it to validate.
It's on brand that it is doing what you want it to do.
Well, that seems like a much more reasoned, rational approach.
Maybe I'm just old and I have perspectives on these things
that don't necessarily align with the rest of the industry.
But here we are.
And I fully agree, Corey.
So from my perspective, and this is why I
want to make sure we're on the same page.
It's not by accident that we called it a teammate.
It's not a co pilot.
A co pilot just watches over your shoulder, does code completion, does, uh, By
design, it's only limited to the human in loop interaction you're involved in.
Here, we're talking about the concept of delegation.
We're saying delegation is new automation.
If you could go ahead and just instruct one of your teammates, To go
ahead and to solve the entire GR ticket queue and to come back to you and
report back with the medium time to resolve every single ticket and trust
that no longer do you need a human to do it, but you only need a human to
supervise the outcome and to make sure that it's fully audited and compliant.
Then you just saved, potentially, dozens of
hours from the human's, uh, week, work week.
And, at the same time, being able to free that human up to do
quite a few more important things that they have on their plate.
So, essentially, go and assign that entire end to end role.
That can also perform in high velocity, high accuracy, and high predictability.
And at the day, be fully audited for compliance reasons.
And of course, that frees up the humans and that frees up,
uh, your, your team's time to, to go and to, to innovate.
I think that that's probably a, a very fair way of splitting the difference.
Now, the obvious question I have that I did allude to at the
beginning, so many companies have seen is, Is this effectively
a three line Python script that starts with import OpenAI?
to be shocked, simply shocked, when that company
doesn't hold still and releases something new.
What is the moat, for lack of a better term?
Well, the complexity of the infrastructure goes beyond this discussion.
Or my acumen as a CEO, to be fair.
There's, we're using over a dozen different language models.
Some of them we're fine tuning ourselves.
Some of them we're training ourselves and some of them
are GPT 4.0 Anthropic, uh, and so forth, but at the end of
the day, everything is broken down to multi agent systems.
So every single operation or task may be invoking a different language model.
Just to give you an idea, Just to understand, if you're going to go
and encounter an operation that requires interrogating a resource and
doing Q& A, that would be invoking a different language model and
probably a different agent than if you would, if you're going and asking
it a question, or if you're going and asking it to provision some.
So you would potentially have, Three different paths you could go
by, depending on the context of what you give it in the question.
And that, for example, you're encountering with the classifier
agent that knows how to classify the right agent that you
would be routed towards, so it can go down that same path.
So, as an example, that's just one element of this.
We can go into different workflows and go into how you go and seek approval.
We have multiple agents, so approver agent isn't necessary
the policy agent and isn't necessary the TTL agents.
So each way you have kind of a Chinese firewall between these agents,
so you can't brute force your your instructions and try to go into get
information you want out of it that otherwise would have been a uh.
under some form of, of, uh, access, uh, control.
There's a lot of, I think, excitement around GenAI.
And I get it.
The first time I saw Chad Chippity do something, it was, it was magic.
It was, oh, wow, I'm watching the future unfold.
And it's rare you get those moments where you get to see it.
Like, it, it reminds me of the first time I walked
to an Apple store and played with an iPhone.
And it was, oh my God, this is so much better
than the crappy BlackBerry I was using.
There are those transformative moments in time.
Now, whether it's worth the massive uprooting of everything
and hurling down the well after GenAI, I don't know.
But from what you're doing and from what I've seen of it, I
think that you're definitely building something interesting.
What that turns into and how that winds up manifesting, I think,
definitely will remain to be seen, but that's the nature of anything.
So I'm sided with everybody who I hate hearing about the next GenAI
company, uh, raising their 200 million round based on a pipe dream.
Then they, there's a huge difference and there's levels to this, Corey.
So there's a huge difference between people putting together
a demo, people putting together a POC from the demo, and then
people go into production and an enterprise grid environment.
And this is effectively where we've already arrived.
We have.
And we can talk about that, but I'm not sure if this is before or
after our embargo, but we have enterprises that are effectively, uh, in
production working with us and enjoying the fruits of their teammates.
Yeah, I think that there are the proof is
always going to be in the pudding for great.
You can tell beautiful stories.
I mean, I love the sound of my own voice.
That's why I have two podcasts, but you can only go so far before.
Having actual customers pony up and saying, Yes, this is valuable.
This is something that we are, we are investing in.
And whether you think it's hokey or not, we're
going to be spending a boatload of money on it.
I mean, Kubernetes is a great example of this.
I thought that was significantly overhyped in some
circles, but everyone's using it at this point.
I was clearly wrong.
I'm wrong a lot.
That, that's the best part about being me.
I know Amazon likes to say leaders are right a lot, but no, no, no.
I like being aggressively wrong.
But then adjusting my opinion in light of new information.
What do you think right now folks are being, I guess,
misunderstanding the most about GenAI's opportunity?
What are they sleeping on that they perhaps shouldn't be?
Without descending into full on boosterism.
Hey, go big or go home, right, Corey?
That's a model.
GenAI, it's a
very powerful technology.
But it's not an end all, be all.
It's not the panacea, right?
You need to have a clearly defined pain that you're going to solve.
You need to have a clearly defined path and
clearly defined architecture to get to that path.
Until that all aligns and all the stars align, everything else is vaporware.
It really is.
And this is where, you know, we kind of
talk about separate the men from the boys.
There's very few companies in production working
with enterprises in, in GenAI applications.
We're one of those.
We're not the only ones.
I assure you there's others that are coming up.
But at the end of the day, the proof is in the pudding.
We give a guarantee on our product.
We even let them opt out after three months if they don't
necessarily enjoy that experience because we are outcome based.
If you're going to go and you're going to enjoy
the fruits of our labor, you're going to pay.
If not, take your money and leave part ways, come
back to us in a year when you think you're ready.
When you think you have a better way of doing it.
Yeah, I have little interest personally in taking money
from people that aren't seeing value in return for that.
It's the, I'd rather lead to a good outcome because then it turns out it's
a hell of a lot easier to sell to an existing customer than it is a new one.
But if you wind up basically leaving them feeling fleece,
you don't really have much of an option to sell a part to.
Word to mouth doesn't travel very well when that happens, right?
No, and what is it, like, bad news travels 10 times faster than good news?
Yeah, I've seen that all the time.
Whenever I'm cynical on Twitter, I wind up
getting an awful lot of, uh, attraction out of it.
But if I say, this is surprisingly great, no one cares.
No one wants to hear positivity.
They want to hear the overwhelming negativity aspect, and we're
now doing our best as a society to algorithmically boost it.
But here
we are.
That's why we've been very cautious not to overhype what we've
been doing until we have proof points and social proof for this.
I'm not going to pick on Devin.
They did an excellent job trying to be pioneers in this space.
But let's face it, they probably should have been a little bit more
cautious before they release their videos and kind of the boosting
about what they're doing, because at the day, they fell where a lot of
companies are falling, being a, having a controllable, Software, autonomous
software engineer requires to have actual controllable measures in place.
I don't think they've done that.
Maybe they will with the 200 million they
just raised, you know, best of luck to them.
Uh, we, we don't have the privilege of raising 200 million, but we have
the privilege of knowing exactly what we're doing and how to go into,
to tame the larger, large language models and to behave the way we want.
I really want to thank you for taking the time to speak with me about this.
If people want to learn more, where should they go?
Well, I guess you can, uh, look at my
shirt, but I don't know if I'm high up here.
Uh, could be I I K U B I Y A dot AI.
Happy to have anybody ask questions.
We, we have a chat bot on, on, on our website, but you could also sign up
because of course you do
waitlist and we're happy to answer your questions.
We have a support channel as well.
So very happy to take questions and.
I appreciate your time.
Of course.
And we will put links to that all in the show notes.
Amit Eyal Govind, CEO at Kubiya.
I'm cloud economist, Corey Quinn, and this is Screaming in the Cloud.
If you enjoyed this podcast, please leave a five
star review on your podcast platform of choice.
Whereas if you hated this podcast, please leave a five star review on
your podcast platform of choice, along with an angry, insulting comment
that I will just assume was written by a malfunctioning chat bot.