Subscribe
Share
Share
Embed
Evolved Radio Podcast: Interviews with technology experts, industry thought leaders, business leaders and other interesting minds. Exploring the evolution of business and technology.
Todd Kane: Welcome back to another
episode of the Evolved Radio podcast.
Today we're talking about something
every MSP is running into right now.
How do you let your team actually use
AI without turning your ticket data
into someone else's training set?
My guest today is Callen Sapien.
Callen is currently building Synthrio,
an AI infrastructure platform built
specifically for MSPs and SMBs.
Before that, he was chief
strategy officer at MSP-Bots.
Here's how I'm kind of
framing this one, Callen.
As we talked about, we spent a decade
dragging clients to the cloud, then we
spent a decade dragging them to security,
and now our clients are dragging us into
AI, whether or not we're ready for it.
And I think we're missing a lot of
opportunities, both on the client end
as well as internally at the MSPs.
So today, Callen and I are digging
into how you actually give your whole
team AI superpowers without blowing
up your PII exposure in the process.
Callen, welcome to the show
Callen Sapien CEO Synthreo:
Thank you very much, uh, for
the warm welcome and having me.
It, you, uh, opened up on everything
that I do every day long, so
Todd Kane: Perfect.
Great guest.
Callen Sapien CEO Synthreo:
Yeah, there we go.
We're aligned
Todd Kane: Yeah.
So, uh, I'll, I'll maybe reframe this
again, like we talked about, um, and
then I'll, I'll kind of set you up to,
to, to, to give us your perspective
on this 'cause I, I think you're
particularly well-suited for this pr-
this, having your experience with MSP
Bots, integrations with PSA, now building
a very, very AI-forward platform, so good
exposure on kind of both ends of these,
these, uh, the, these, uh, this topic.
And the part that concerns me is I see
sort of this multi-layered approach
for people, especially service managers
that I interact with a lot in the
work that I do, and some of them are
really leveraging AI in great ways.
They're, uh… In particular, I've
seen, uh, a lot of people dumping ticket
data into a model and having it do some
analysis and, uh, spotting some trends and
giving it some insights, and that stuff
is golden because doing a service review
manually is time-consuming and requires
a lot of sort of mental bandwidth and
energy to kind of connect those dots.
So being able to offload that to
a model is super, super valuable.
But rightfully, people are a bit nervous
about where they give that data to, right?
If you're gonna export all of your
tickets, just dump it into a model and
say, "Give me some insights," you gotta
be pretty particular about how you're
actually servicing that model, what
information potentially goes in there.
Simple things like the fact that, you
know, I think you and I, you and I talked
about this, is like a lot of people
don't even recognize, like even if you
have a paid model, there is a setting
in your options to turn off whether
or not it goes into training data.
So just because you're paying
doesn't necessarily mean that
you're clean to give it kind of
as much information as you want.
So I recognize why people are cautious
about this, but I see this as sort of
a tale of two MSPs, the people that are
leveraging AI in order to be able to
do these things and hopefully doing it
well, versus the people that are not
able to take advantage of this because
they're somewhat justifiably fearful about
the best way to take, to go about it.
So what's your, what's your take
on how we go, we go about this in
a safe fashion in order to maximize
AI's potential inside the MSP?
Callen Sapien CEO Synthreo: I, I, I--
That's a great, uh, overview and, and I
break it down into really three areas.
There's the technological, right?
Did I uncheck that box?
Uh, I have ZDR enabled,
zero data retention, right?
Uh, uh, we love to throw out
acronyms in our space and
Todd Kane: It's a new one for me.
I like it
Callen Sapien CEO Synthreo: and,
and zero data retention is actually
probably the most critical piece.
I'll circle back to it because I don't
wanna, like, do the ADHD thing where
I, I, I go off and I say I'm gonna
talk about three things and then, then
jump down, which I almost did just now.
So you've got your technical,
which includes things like
zero data retention, ZDR.
That includes the, the, the actual
checking of the settings, uh, and the--
and then the secure, and we've been
talking about this for a long time,
whether it's cloud security or even going
back to the aughts where it's the right
permission levels, uh, set up, right?
The right identity and access policy.
Uh, so there's the technological side.
Then something that we, we overlook
a lot is the contractual side.
We, we have gotten used to these,
like, giant SaaS agreements that we
just, we just click, "Yep, I read it.
Yep, I read it." And most courts have
said, you know, it's not really a, a--
truly something that we would expect.
It does limit liability, but, but you
can't enforce every clause, right?
Uh, but the AI, uh, terms are actually
not as long as most SaaS terms,
and they have some hooks in there.
Uh, and then the last
piece is cultural, right?
Um, y- y-- we, we are-- You, you
mentioned the, the push, push, now pull.
We cannot suddenly, three years later,
come in and say, "I've been handling
your security for a really long time.
I ignored AI.
I just told you to ignore AI.
I know you didn't ignore AI.
Now I'm gonna shut everything down,
and you can't use it," because
people have been getting value.
Even if that value has been a picture
of my fridge and see what I'm gonna make
today or what's in my fridge, they're
getting value, and they haven't had the
pain as much as they have in other areas
that they haven't had the governance.
And so one other thing that's kind
of interesting on that last part is
I was on with an MSP in Canada and,
uh, the, the Canadian AI Privacy
Security, uh, Act failed, right?
It did not go through.
But that doesn't mean the FIPS
Act that, that exists around data,
data retention and all of those
things can't be enforced, right?
GDPR counts, FIPS counts whether or
not it was done by AI, SaaS or a human.
And we, we-- when we're, when we--
we do need to bring that in as a
governance piece, but, but those
are the three kind of buckets.
Going back to the zero data retention,
because I do think that's the most
important one, second is the contractual,
but the, the, the zero data retention
is critical because you can uncheck
that box that says or check that box
that says, "Do not train on my data."
There is no box with any vendor right
now that says, "Do not retain my data."
And they propose in their contractual
side that they have legitimate reasons
for it, if you're r- abusing it, if they
need to see if a model's misbehaving,
if they have these other things.
But every single model provider
asks for or demands as part of the
use of the product, the ability to
keep data for as little as seven
years, which is a long time, uh,
and as much as, uh, indefinitely.
When we look at like Google having shifted
from don't be evil for the first 11 years
and, and listening to, "I'm not gonna
train on your data, I'm not gonna do
these things," and still work within the,
the confines, we know that if somebody
has access and they keep access, it
may not always be used for good things.
I don't wanna be like a scary, Anthropic
is using your data to destroy the
world, but, but if they retain a copy
of it, you know, things can change.
But I, I-- Before I jump into
the contractual, I'd love any
thought that you have there.
Todd Kane: Yeah, I guess like the biggest
issue with that is gonna be breach, right?
Like your data is retained somewhere and
it's not necessarily-- 'cause originally
like when the, the, when GPT was first
sort of exploding and there was like
these stories of like, uh, these guys
at Samsung that loaded up a bunch of
information in-into GPT and, you know,
uh, nearly eliminated one of their
patents because they'd, uh, in ess-in
essence made the information public
into one of the, one of the models.
Or like your data kind of resurfacing
as information for other people if
it's a part of the training data.
And I think a l- some of that is
definitely, uh, not as much of a
concern as it originally was, and it
goes to sort of the f- like the first,
uh, I would say edict of AI safety
is don't use an unpaid model, right?
Like the free model is there
for training data, right?
So if you're gonna give your data
to anything, make sure it's a paid
model at the very least and you're
turning off some of this data.
But to your point, like
it's still there, right?
Like there's some websites even for myself
where I will tell it not to save my credit
card data 'cause I'm like, "Yeah, it's not
really that big of a company if it's not
using Shopify or Amazon as a backend cart.
Like do I really trust it?" Same idea.
Like you're, you're essent-uh,
essentially kind of giving this, uh,
this information over for free forever.
Uh, so yes, there's a risk of data breach.
Some people think like, um, "I'm
a small player, you know, my
information, you know, would it…
w-why would they possibly use my
information for something," right?
And it's not really that.
It's like, okay, data exfil, they dumped a
petabyte of information from, uh, AI model
X. Your information's in there, right?
So I think that's sort of the, the biggest
risk is, is just data exfil from some
scary event in the future and, you know,
could be even sort of strangely benign.
Like we had these conversations recently
with Alex Dao on, on security where, uh,
training models broke into other companies
in order to gain access into stuff, and
that's exactly how this could go down
is like, you know, uh, Fable Six, uh,
launches an attack because it needs more
training data and starts breaking into
all the other AI companies and s- and
pilfers all of its information, right?
Like, uh, you never know how
this stuff's gonna come about.
So I think that, that's a good
point is just understand sort of
where the data is resident and
how it's being utilized right now.
Callen Sapien CEO Synthreo: That's,
that is a, uh… And that, and that
even in-- So there's a large problem
of people being incredibly empowered.
Uh, uh, we, we, we have the ability
to kinda host your own MCPs, right?
And, and that may seem somewhat
superfluous because there's a lot of
MCPs out there, but the reason we have
it is because of the data residency and
the other pieces you're talking about.
I've got a, a partner that needs data
residency, uh, from regulatory reasons.
Uh, they can use the official app in the
store that exposes financial information
and these other things, the official MCP
for a marketplace, uh, through Claude.
But Claude won't guarantee that that
data doesn't on US servers or it's
saved there and these other things.
People need to understand what that flow
of data is, and it's actually a tremendous
amount of value that an MSP can provide.
Uh, it's a huge value add,
giving this visibility and
building the services around it.
And at the risk of sounding anti-security
again, and I'm not anti-security, I've
been a CISO, but at the risk of it, uh,
the thing that I would like to actually
point out related to this is you can't
sell and protect AI the same way that we
did endpoints and other pieces because
it doesn't operate or act the same way.
And in the last five years, six
years, done a lot in security, but
a lot of MSPs that thought they were
gonna be MSSPs did not become MSSPs.
They bought a SOC, they bought
an MDR, they bought an outsourced
helper to deliver these things.
So they've outsourced the services
side of this instead of standing it
up because it costs a lot to stand up.
the, the challenge with trying
to sell AI that way is the
management is where the margin is.
The, the, the, the driving of the
output, the guidance that, that's
required to actually impact the P&L
is where the management is and the
guidance of, and that, that actually
shifts into the contractual side.
In the US, a lot of the court cases
are up in the air still on whether
something like attorney-client privilege,
if someone's using AI, counts, right?
Because for the same
Todd Kane: Yeah.
Callen Sapien CEO Synthreo:
destroying your patent.
Todd Kane: Mhm
Callen Sapien CEO Synthreo: And the
one thing through the cases that are
pro and for and against and this and
that, the one commonality through all
of them when we look at what's becoming,
uh, stare decisis, the, the let the,
let the decision stand, is who had
the governance and the stewardship
of the data contractually, and was
there anyone that could take this data
for a legitimate or an illegitimate
reason and shift it, uh, and use it?
And, and that's where the
contractual side comes in.
Pretty much every main, main provider
that's out there that we're going to use
this data to, to, to, to learn something.
Not train the model, but we're gonna
learn something, and we can access
it over the course of using it.
And so that, that to me is a, a
big area that we can get into.
Todd Kane: Okay.
Um, yeah, I think the, the governance
piece, like that's, that's I,
I think really interesting.
I'll, I'll, uh, governance, I'm gonna
write this down just so, 'cause we
got-- we're gonna open up tons of
ADHD threads here as we go, I'm sure.
All right.
Um, the, uh, I guess like the function
around sort of the util-utilization for
PSA data, like that, that to me, like I
said, is like I, I wanna focus on this
because I feel like there's an incredible
amount of value for it, but it's such
a fraught issue for doing it correctly.
Um, so obviously, okay, you
know, we're not using, um,
a train, uh, uh, free model.
Maybe we're, uh, handing
it over to a paid model.
Uh, I think understanding, uh, where the
contractual I think lays into this is
like being clear with the client that
maybe you're using their data, right?
Like if you're supporting a law firm
and they have client information,
uh, potentially in some of those
tickets or the connections that
you create to them, that creates a
bit of a, a slippery slope as well.
And maybe that's an interesting one as
well, is how do we think about the PSAs
that have AI capabilities built into them
or other tools that are connected, right?
Like that is, you know, you're not
even necessarily giving something to a
model, but the model is passing through,
or sorry, the data is passing through
another model in some fashion you don't
necessarily have direct visibility to.
So like, and I have to imagine this
is incredibly common with most of
the platforms creating some type
of AI capability, whether or not
it's a third-party module or just
built into the system, right?
So how do you think about that piece?
Callen Sapien CEO Synthreo: Yeah,
I think it's, it's, it's, it's
a completely under-addressed
problem that exists right now.
Uh, it, it-- The, the, the, at least
the use of AI within traditional SaaS
products in a completely unmanaged way.
Monday.com, for example,
added a feature that basically
put Lovable or Replit in it.
You can build apps, build MCPs, connect
things into your Monday as if you
were building, building front ends.
And that's not… That's-- It's
very, very challenging to be
able to get visibility into what
people are doing in, in that area.
And so I think that, uh, it, it's
still kinda coming out in the wash.
We're working on some ways
of building visibility.
There's some really cool apps out there
doing that, but it's, it's, it's, it's,
it's very heavyweight right now, and
it, and it's across every single app.
Uh, when you look at a standard MSP, they
use 17 tools to deliver the, the output.
Their customers use about 23 to 28 tools.
All of them have AI in them now
processing it, and most of them
have updated their terms to make
it an opt-out, not an opt-in.
You're, you're, you're not… It's
not like calling in used to be.
I think it eventually will be, but, but
it, but the data is being used, maybe
not to improve things, but it's being
pushed through a model that, that, that
we don't know what it's gonna do with it.
When we get back to kinda your core
question, which is, you know, how
are these things being used, and how
are we able to do it in a, in a core
way or in a, uh, in a strong way?
One thing that, that I think we talk
about a lot, and it's shifted with
AI, people have asked me why I think
that AI is so good at coding, right?
This was a hard task that
required a lot of brainpower and
a lot of reasoning that exists.
And the whole reason that it's great at
coding is because open source software
exists and, uh, Bitbucket and, uh, and,
uh, and Stackbucket and Stackjack and all
these, these things where I could go look
at good code examples, bad code examples,
and they train the models off of that.
I think right now we're trying to protect
the wrong things with our data, even the
PIIs and the other, the other pieces.
We have to do that because of compliancy.
But the data Your MSP's data is not
very different than this MSP's data.
That lawyer's data is not very
different from that lawyer's data.
But what is different is
your processes and the how.
You're seeing more and more, uh,
these attacks almost, uh, they-- I, I
would actually say they're almost to
a level of attack where, uh, Claude
Design's a perfect example of this.
Claude went and had a Figma integration.
Claude learned how Figma works, and
then they launched Claude Design.
If someone doesn't put any PIIs in there,
doesn't put any data, doesn't put anything
in there around what the ticket actually
contains, but they, they actually describe
their process to the AI, now I've given
the AI what makes me a 28% EBITDA MSP
instead of a flat or an, a 5% EBITDA MSP.
If I put that data on how the, the,
the process of how this lawyer treats
clients and, and runs through their
process, now I've given away my secret
sauce, and anybody who uses that model
that it's improved itself on gets that.
And I think that's one of the things
that we really do have to, we have
to talk through, and, uh, how do we
protect the knowledge and the how?
Because IP is, is, is shrinking
as a, as a capability.
Um, and that's something that I, that
I think that is very interesting,
and how do we protect that?
And that actually goes
back to that ZDR, right?
If, if the model doesn't remember
anything about what happened,
then, then it can't learn.
Todd Kane: Yep.
Yeah.
I, I almost feel like that one's
unavoidable, and I almost feel
like, uh, also, uh, like one of my
favorite expressions that I've said
forever since I've been consulting
is, "Knowledge is easy, execution is
hard." Uh, I don't know that there's a
lot of proprietary information in the
future that we can actually protect.
Uh, uh, outside of things like
the Coca-Cola recipe, right?
Like sure, like there's some s-
specifically proprietary information.
But I think it's, it's a good example
of what is really strange about the MSP
industry that I call it a lot is we all
fundamentally have the same business
model and noob- no two businesses
run even remotely similarly, right?
Like, a- and this is despite the fact
that like there's a huge incentive and
a huge model for commoditization and
forcing people to, to work the same way.
But quite frankly, most organizations
are a reflection of their owner, right?
And anyone who has never noticed this,
just like do a bit of a tour and think
about the places that you've worked,
the companies that you've looked at.
They are always a reflection of
the owner in this weird, weird way.
So I think that's really only
the, the only differentiation.
So I don't know that like the
information of, uh, how I do something
is, is necessarily as sort of a
secret sauce as maybe, maybe it
would be, especially in the future.
Because knowledge will just be
so systemic and available, right?
Yeah.
So,
um-
Callen Sapien CEO Synthreo: I really
like, I'm not a huge Sam Altman
fan, but I do think he, he's been
right on a lot of things, already
built a trillion-dollar company.
Uh, but, uh, and a
trillion-dollar non-for-profit.
Uh, but, uh,
Todd Kane: If you're getting
technical about it, yeah.
Callen Sapien CEO Synthreo: you're
getting technical, uh, he, he said
that AI, and it's, it's turning into
hands as well, but he said AI is an,
is a utility, but it's intelligence.
It, you, instead of electricity,
instead of, instead of gas or, or
these other things, it's intelligence.
And you turn on the spigot, and
companies that can spend a lot like
they would on electricity can spend a
lot on intelligence, and companies that
can't will get a little intelligence.
And I think it, it actually bodes to
your point of the fact that it, it's
going to be ubiquitous, and there's,
there's a, a, a time that we can
protect it, but it is gonna go away.
It's just something to be
aware of as we go through.
When we actually talk about the
practical application of using AI
in ticket data, I think that there's
still a lot of focus on personal
productivity, and we haven't really
s- got into the infrastructure phase.
That's where we're seeing the, the
labor, the FTE, and the P&L impacts is
when we start to get into the actual
infrastructure phase of installing
an agent, of installing, uh, the data
layer, the ontology layer, if you will,
uh, within an o- an organization so
that we can delegate and dispatch to,
to the, to the agent to get it done.
I actually think that human in the loop
is becoming a more and more antiquated
term, and it's almost a Luddite type term.
It, it, it… Human in the loop,
humans make mistakes as much or more.
Uh, there's, there's a cool company
in our space, uh, that got me turned
on to, uh, AI, uh, before it was cool.
I was still on the machine
learning and data as well.
I was at, at MSP-Bots, and we were trying
to figure out AI and machine learning.
But, uh, Mark Elaev, the fou- one of
the founders with Matt over at, at, at
Thread, uh, called me in March of 2022,
and he's like, "We've got this thing that
we got into the beta of called OpenAI,
and we're, we're using it to, to, to guess
and accurately put in time entry against
a ticket." And I was like, "Oh, cool.
How accurate are you?" Like, "84 or 85%
accuracy." I was like, "That's really
high." And people didn't adopt it because
it was 84 or 85% accuracy attacks,
uh, remember these tickets had zero,
Todd Kane: Yeah, exactly.
Callen Sapien CEO Synthreo: entry
Todd Kane: an 84% improvement, yeah.
Callen Sapien CEO Synthreo:
It's, yeah, and, and the ones
that did are 37% accurate.
Todd Kane: Wow
Callen Sapien CEO Synthreo: you know,
the, the, the level of which I, I get
into debate a lot and people argue with
me on, and I understand accountability
is the hard part, but, you know, with
like self-driving cars, they are 100%
more safe than human-driven cars.
But that isn't good enough
for a lot of people because
can't hold the car accountable.
But you can hold the company and
the machine and all these other,
just like if a seatbelt fails.
Um, I think the, the, the answer
should be it should be better than
the human in a measurable way, and
that's what I think when we're doing
these things we should be looking
at within the, within the instances
Todd Kane: Yeah.
Tired of fighting the MSP fires alone?
The Ops Leader Pro group connects
service delivery professionals who
understand your daily challenges.
From KPIs and workflows to career
planning and team management, Ops
Leader Pro has systems for you to use.
Join operations leaders from successful
MSPs who are sharing real solutions for
managing client expectations, optimizing
service delivery, and making your service
delivery team as effective as possible.
Ops Leader Pro, 'cause your service desk
deserves more than just survival mode.
Visit opsleader.co.
That's O-P-S leader.co to apply
to join the public community.
Todd Kane: All right.
So you mentioned, um, uh, sort of like
the where the information is resident,
and this I-- this is another layer that
I think is really fascinating about
this is, okay, so yes, you could use a
cloud model for some of these things.
Um, you could use a hosted like Azure
container model for, for these things.
Callen Sapien CEO Synthreo: Mhm.
Todd Kane: your guys'
approach is different.
It's sort of like, uh, maybe
a hybrid between the two, if
I understand that correctly.
Yeah, can sort of private
containers, uh, uh, type approach.
Um, the other one that I was thinking
is like, well, maybe are we just
gonna start doing a lot more, uh, ho-
self-hosted and private models, right?
And, uh, like that kind of makes
some sense because I feel like, like
despite the fact that the frontier
keeps on pushing forward, like the
frontier models from Anthropic, OpenAI,
uh, you know, Kimi, they're, they're
advancing and their capabilities
are, are, uh, advancing incredibly.
But they're also like some of, some of
this is diminishing returns on where it
is right now is perfectly fit for all the
things that I continue to need, right?
And I am like this close to
moving to a private model, right?
Probably the next, uh, the
next Mac upgrade that I get,
I'll go, I'll be fine, right?
Like I won't necessarily need a frontier
model for most of the things that I do.
But, um, so I was thinking to myself, so
like maybe most of these companies should
just maybe look at self-hosted models and,
and for, for some of this information.
Won't be as fast, but you know, if you're
just doing some data analysis and let
it churn for an hour, then great, okay.
It spits out something relevant.
I was having a conversation
with, uh, uh, John Dobbin.
I'll give him a shout-out for,
for the, the insight on this.
Callen Sapien CEO Synthreo: Yeah
Todd Kane: he scared the
crap out of me on this.
He's like, "So here's the thing.
Most of the self-hosted models, like
the open source models, there's,
there's very little guardrails and
containers built around them," right?
Like the frontier models have a lot of,
uh, capabilities around prompt injection
that they will not do things that are,
that are gonna be nefarious, right?
But like, uh, what, what the, the sort of
the, the idea that he, that he suggested
is, say you have like a website where you
can have clients submit tickets, right?
And someone goes in and submits a
malicious tickets that gets processed
by a local AI that doesn't have good
prompt injection, uh, management
on it, and now all of a sudden
it starts exfilling data based on
that, that, that ticket submission.
I was like, "Oh, crap. Okay.
Private hosted models may not
be the best play here," right?
So it kind of scared me of like,
there's, there's so many risk
and reward trade-offs between the
different approaches here, right?
Callen Sapien CEO Synthreo: Mm-hmm.
Um, no, it's spot on.
There's, there's, there's really
two challenges currently with the…
Outside of the fact that you gotta
probably spend truly to, to, to get
something really meaningful, you
probably spend eight to $10,000 just
with component costs right now to, to,
to get something that, that can run.
And then, and then you've got
energy costs and the, the other
pieces because GPUs are heat.
Uh, we, we, we, we went through
this with the mining era of
Todd Kane: Right
Callen Sapien CEO Synthreo: right?
Uh, and, and so we, we know that, you
know, there's probably gonna be a lot
more solar panels popping up, uh, for, for
people in a little while, or geothermal.
Uh, but the other piece to the
local models, I, I do agree with
John that, that, that there are…
But you can, you could put the
same type of guardrails, right?
They do have lower context windows,
right, which is a problem, uh, because
you fill up pretty quick and you don't,
you don't know what you can attack.
Uh, but I do think that is, this
is absolutely w- going to be part
of the solution in the future.
Uh, but one of the big challenges
with these models right now
is the hallucination rate.
Uh, we see with locally hosted models,
uh, in the 40s, 50% hallucination rate
over the course of a conversation.
And so you, you really need a lot of
human intervention, both from the security
side that John was talking about, but
also from the business outcome side.
The, the models are more prone to
hallucination because the math isn't
as strong, and the context is worse,
and the memories are not as powerful.
Todd Kane: So that it,
Callen Sapien CEO Synthreo: really where
Todd Kane: the, the, uh, let me dig on
this 'cause like that, that is a function
of smaller context window by nature
because of limited infrastructure and
something to do with the models themselves
or sort of the parameters that are built
around it, or just the context window?
Callen Sapien CEO Synthreo:
It's, it's, it's a combination
of, of kind of all the pieces.
The smaller models with
the lower parameters have
Todd Kane: Yep.
Callen Sapien CEO Synthreo: capacity,
Todd Kane: Yep
Callen Sapien CEO Synthreo: what you said.
Uh, you know, to put another
way, it's like, you know, using,
uh, less of your brain, right?
A lot of those models were
designed to have, for example,
like 32 billion parameters that
they can take on at a given time.
We neuter them down to seven
to eight billion parameters.
So you've got a prediction engine
that is trying to predict something
with a quarter of the context and
the memory capabilities that it has.
So it does work great for, for certain
repetitive tasks, QA, processing of,
of, of almost a deterministic behavior.
Uh, but where it, where it really
starts to, to fall down is when we, when
we start getting into, uh, you know,
adding in multi-step processes, right?
Uh, multi-step is really the big part
that, that gets hit, and that's where
we can add in different workflows
and loops and, and these other
things, but it, it definitely…
It's actually one of the main reasons
why John's security breakdown happens
as well, because how you overwhelm and,
and overcome the guardrails largely
with these models is either by injecting
a, a, a, a, a mistruth and calling
it on it, and then, and then now it's
like, "Oh, I was wrong on this. I
lied, so now I'm gonna relax my other
things because, because this is here."
Or then context bombing.
So I'm gonna drop in 50% of my context
window because now it's the guardrails
were maybe 30% of every turn in the
conversation for the first half of
this conversation, now they're 18%.
So as that math is, is, is expanding,
the, those, those guardrails just
have less weight in the conversation.
There's still more weight than the,
than the, than the attack, but they
have less weight overall, and then it's
just about consistency in attacking it.
Todd Kane: Especially given this…
Callen Sapien CEO Synthreo: bel-
Todd Kane: Go ahead.
Callen Sapien CEO Synthreo:
No, no, go ahead
Todd Kane: Well, I, I think it's,
that's relevant specifically for sort
of the use case that I'm advocating for,
in, in particular is ticket analysis.
So you,
Callen Sapien CEO Synthreo: Yes
Todd Kane: say, even s- three months of
data, that's still a lot of data to dump
into, uh, like, that's certainly not gonna
fit in a 200,000K, uh, window, right?
Or 200K window.
Uh, e- even a million, I think, like, it
might fill up a fair bit, but, you know,
you gotta be sensitive to that, right?
Callen Sapien CEO Synthreo: We, we, um,
we're handling this in a, in a way where
we're tr- what we're trying to do is,
is not offload the, the model itself,
but offload the compute, because that
will reduce some of the, the, the cost.
And so what we do is we try and keep
the thinking and try and keep the
actions local, because that's, that's
more controllable and definable.
Uh, and, and then batching.
I, I think batching is highly
underutilized by practitioners,
and this is where the Ticketdata
side can come in very well.
If I can, if I can run through a
recognition pattern, whether that's
machine learning or whether that's a
SLM or whether that's just a model,
and say, and categorize these things,
then I can send them through in
batches, and then I can reconnect.
Uh, Gary Vee just, uh, said
something a couple days ago
that I really, really liked.
Um, and, uh, I've followed
him for a long time.
I didn't expect him to become an
AI guru, but, you know, the guy's
a guru in so many other areas.
But he calls it the 15/85 rule,
and he, and he started it with his,
with his, like, leadership team.
He wants to be involved in the
first 15% of the planning and
strategy, let them go off and do
80%, and then back there for the 5%.
And I think that when we apply
it to AI, it's very similar.
Let me be part of the planning
and the strategy, go do your work,
uh, and then, and then come back.
And we could actually probably localize
those much better than, than otherwise.
I think Zofik was on, was on that path.
I d- you know, I haven't talked as
much to Lee since their acquisition,
but, but I think that was really
what he was trying to solve for was
because we had people running these
local models, and so he, he took that
on th- uh, upon theirselves to do it.
Todd Kane: It's funny 'cause like this
morning actually in my group coaching
session, I, I, I talk about this
model a lot, and it's 10 / 80 / 10.
So, you know, he's got the,
Callen Sapien CEO Synthreo: Yeah.
Todd Kane: the 50 and 80 and five.
Uh, so yeah, my model is, is 10 / 80 / 10.
I don't remember originally where I got
this from, but it's, it's an industry,
uh, sort of leverage productivity
model, especially for delegation.
This is where like I originally
started using this in, uh, uh, avoiding
executive swoop and poop, right?
Because the, the executive would sort of
get in there and mess around with stuff,
and they didn't set up the parameters
for what the delegation was or what
they wanted, so they would make a mess
and then sort of show up at random
times and not pr-produce any value.
So I would tell people like, "Okay, use
the 10 / 80 / 10 model for delegation,
10% l- front loading expectations,
delivery, what you wanna protect
against, then let them go and do 80%
of the work and then 10% cleanup on the
end." So exactly the same, same model,
slightly different, different numbers.
But where I think this is
particularly useful is delegation.
Like that's where I talk about
this framework a lot, right?
And what-- I think what you're pointing
to is we need to start thinking more about
agents as independent, sort of autonomous
employees doing particular work.
And it's weird because like it's not
necessarily contiguous, but I think
we still wanna think about it in a
10 / 80 / 10, uh, delegation exercise
of like, "Here's the parameters.
This is what I want.
This is the expectations.
Don't do this, do this,"
all of those things.
Let it churn for 80% and then
inspect it on the back end, right?
And like I'm sure you've heard this term
meat proxy, um, which is, uh, uh, people
that just take whatever comes from AI
and turns around and, and spits it out.
Uh, and this is where like work
slop is coming from, right?
The, the whole idea of, uh, you know,
I, I, I was gonna send an email.
I had ChatG- ChatGPT punch it up to
like a three-page or a three-paragraph
summary with lots of information and data.
Uh, I emailed that over to somebody else,
and then that person uses their AI agent
to digest it into like the f- the, the
much smaller version of this, right?
So it, it's just meat proxy
to meat proxy, and we're kinda
uselessly using AI in, in between.
Uh, so now I'm just ranting.
But the, I think the, the, the, the,
the way of, uh, using delegation
frameworks for AI and the similarities
that we should be applying in
general, uh, sort of collaboration
and management is very on par, right?
Callen Sapien CEO Synthreo: I, I,
it, it actually uncovers a very human
problem that we're dealing with right
now, and, and, uh, it's-- there,
there, and there's two pieces to it.
We have looked at leadership as
a… And I mean leadership and
managerial, because you don't have
to have reports to be a leader.
Todd Kane: Mhm
Callen Sapien CEO Synthreo: looked at
that as there's a class of people that
have earned that right, and we're going
to invest in those people because we think
that they would be good leaders of humans.
Uh, in order to be a powerful AI user
and actually use it in a way that is
going to impact in a positive way your,
your clients, yourself or, or your
mission, you have to be a good delegator.
And we've gatekept these things to the,
to the, to the upper class of this.
And including, we've told people
like, "Well, I don't think
you're cut out for leadership."
Well, now you have to be a leader.
And I think it's going to-- And we--
I've actually seen a rise in more like
HR consultants going into leadership
management, teaching these things.
It's also, uh, been a challenge for
people that have, uh, globalized
talent because we ma- we, we globalize
the talent and we, we, we offload,
workflows and intellect to individuals
to follow a process with no deviations.
And, and so, you know, comp- countries
like the Philippines, uh, certain areas
of India, uh, Vietnam, Indonesia, uh,
Pa- Pakistan, these areas have, have
built, you know, farms and, and, and,
and cities mul- you know, um, uh,
of, of people that, that are going to
follow these things, and if they don't
follow it every step, they're fired.
And we're asking them to adopt the
management capabilities and the
creativity that we've trained out of
them for 45 years as we've outsourced.
And it's, it, it-- and a lot of MSPs use
globalized talent, and it's something
that, that we're going to really have to
address in a very short amount of time.
Uh, otherwise, the token cost is not
gonna be worth it, in my opinion.
Todd Kane: Yeah, and that's the other
issue that I think we're facing here.
Maybe, uh, slightly off track,
but, you know, tangen-tangentially
related is, um, the over-indexing
on sort of the capabilities of AI
and where they can actually fit and,
you know, the token cost for those.
People are now going, uh, you know,
uh, it was sort of the last six months,
eight months was token maxing, token
maxing, we're gonna do amazing things,
20% layoffs, and then they're like, "Holy
crap, why are our costs up 70%," right?
And so they were token maxing.
Now it costs more than the people that
they had sort of doing jobs before, and
maybe not achieving the same ends that
they thought coming into it, right?
So I think this will be an interesting
trend to watch, but I, I think
what you're, you're alluding to is
more of sort of a business process
thing of like, how do we, how do we
compartmentalize and really understand
the work that we're doing, and how do we
intelligently hand it off to workers or
digital workers in a way that actually
produces the results that we want?
I think that's gonna be a really
fascinating exercise going forward, right?
Callen Sapien CEO Synthreo: Yeah, I
mean, that's i- in the-- We're, we're,
we're coming up on a year officially
really in market, uh, in, in November.
And in that year, we've had
really two pretty dramatic shifts
onto my company's identity and,
and the problems that we solve.
There, it's the same mission, but, uh,
technology and the landscape has shifted
so much that, that we've realized that
we need to be, uh, the optim- the, like,
the, the greatest optimizer of delivering
those outcomes for the business class.
So what I, how I would, how I would
liken that is basically like Cursor for
delivering results with AI, Uh, and,
and Cursor does deliver results for AI.
It's just specifically around coding.
And so we've, we've built it, and it's
to the point where I think you mentioned
the diminishing… No, uh, I don't think.
You did mention diminishing returns.
And we can get out of a, a, an open
weight model like Inkling similar
to Fable 5.0 results, uh, with,
with our Wingtip engine, right?
And, and the cost of that is a 10th
or, or a 40th, or if you're willing
to use a GLM, maybe a 100th to
get, to get 90, 95% of the output.
The other piece is that infrastructure
piece, and I think it's a, the,
probably the biggest opportunity
with AI, though, for the MSP
market, is we gave up infrastructure
when we, when we went to 365.
You've been in the industry long enough.
I-- fortunately, cybersecurity came
quick enough, but I remember talks of
layoffs, panic in 2016, 2017, as we moved
from, "We own all the infrastructure.
We're building data centers," to,
"We're getting 20%, 10%, 8% of what
Microsoft will give us, and, and
now we've got to survive on this."
Then security came in, it saved us.
Uh, and but we're now back in the
spot where we get to manage and
maintain the infrastructure again,
and we need the right people that
can figure out how to do those tasks.
But then we need to figure out,
like, where does that live?
What does this look like?
And how does that scale?
And it's a huge opportunity, though,
that, that exists, uh, in, in the space
because I think we can get back to
the 60, 70% margins and have partners
that are absolutely willing to pay it
because they're getting so much value.
But it requires exactly what you
said, the right delegation, the
right harnesses, and the right,
uh, infrastructure to, to do that,
Todd Kane: Yeah, and that's
the biggest thing here is, is
Callen Sapien CEO Synthreo: at.
Todd Kane: yeah, there's always where
there's complexity, there's margin, right?
And we're now getting to the space
where like y- you can't just sort
of, "Oh, just go get ChatGPT," right?
I think that there's sort of like layers
of where MSPs are going wrong in their
AI strategy, and I've been ranting about
this as like that's why I sort of frame
it as like pulling their clients to cloud,
pulling their clients s- to security.
Like, where are you guys on AI?
You're not leading, right?
Like there's so much shadow
AI going on, it's insane.
So first there's just the
governance aspect of that I
think is a massive opportunity.
Um, but I think like the, the work that
you guys are doing is a huge opportunity
of actually stepping into, you know,
the industry term Pax8 is, is sort
of, uh, advocating for is, um, is, uh,
managed intelligence provider, right?
And right now, a lot of people are just
like, "Here, let me set you up with
Copilot." And it's like, well, okay,
like that's easy, but like that's not
gonna produce a ton of value for people.
I honest- I kind of firmly believe this
is why we're seeing so many damning
stats in the industry of AI initiatives
failing in enterprise businesses,
'cause they're just like, "Here you go.
Here's AI.
You're gonna do great," right?
Like, "Here you go, Tommy." And like
nothing comes out of that 'cause they
don't know what to do with it, right?
Like, no one's trained them.
They don't know how to use
those models beyond just asking
for recipes and stuff, right?
So the, of course, that's
not gonna produce anything.
The second layer would be, you know,
"Hey, let me do consulting with you. We'll
s- we'll set up, uh, like so a Claude
account or an enterprise GPT account
and get everyone sort of rolled out on a
maybe a arguably a better model," right?
And maybe this is the way to go about it.
And now it's just consulting with
reselling kind of a SaaS product.
But I think what you guys are doing is,
is much more the hybrid of like, "I'm
gonna give you a console that allows
you to do like open weight routing on to
different models and give you the privacy
protection and, and the portability to
multiple models." And I think this is
massive 'cause of your point, like when
we were, we were talking before, maybe
you can d- you can dialogue on this story
a little bit of like you had a client,
like they were paying 50,000 a month
in self-hosted Claude plus Azure, and
you guys slashed that to like just over
10,000 a month with, uh, with, uh, the,
the platform that you guys are doing.
So I think this is the value that you
can bring to people is like, not only
am I giving you the capabilities for a
fraction of the cost, but I'm actually
able to manage it for you rather than
just sort of like, "Okay, hang on, let me
try and log in as an admin to your, to y-
to your, your, uh, your profile," right?
Maybe you can expand on sort
of how you guys are doing that.
Callen Sapien CEO Synthreo: Yeah,
I, I think, you know, the most
forward-looking, that we end up
encountering are dealing with actually
some of the initial 365 problems, right?
The federated access
into all these portals.
We-- There was a partner of ours that, uh,
they were wor- they were using N8N, and
they, they were around with my co-founder,
and they're like, "You know, the, my first
thing I do every day is log into 38 N8N
portals and see if everything's okay."
Todd Kane: Ugh
Callen Sapien CEO Synthreo: It's like,
that is, that is not… I mean, like
none of us got into… I mean, maybe
somebody got into it to do that type
Todd Kane: Some, some automation nut, yeah
Callen Sapien CEO Synthreo: Yeah, yeah.
There, there are some you know,
that, that exist in the world.
Uh, but, uh, and I, I don't
want to yuck anybody's yum.
Uh, but, uh, but at the same time,
that isn't sustainable, right?
And, and you're gonna miss something,
and something's gonna crash, and,
and, and we're, you're gonna miss it.
And, and so for us, we, we do look
to, to, to bring that You said
it, manageability, guidability,
governance over, over the piece.
W- w- it's interesting 'cause we
also had a, a recent, uh, partner
that is a Claude network partner.
They're, they're reselling… Not
really reselling Claude, but they're
reselling a service around Claude.
And they had a, they had a partner
that said, "You know, I'm, I'm, I'm
over." You know, we started this
conversation by saying, "Hey, a
new model might have dropped," uh,
and, and because it all sucks now.
And, uh, and they, they said, "You
know, I'm tired of this, this rat race.
I need to go to something that's
agnostic because if I pick a winner
right now, I'm gonna be jumping every
six months to try and get the most out
of this stuff." And they said, "Okay,
so what do we do with the last eight
months that we've been in Claude?
Do, how do I export this work?
Where do I put my artifacts?"
And, you know, the MSP's like, "I've,
I don't know. We'll, we'll export your
chat history that we can, and we'll, you
know, see what we can do about your, your,
uh, your projects." But the portability
issue and the, the, the other pieces that
people are just starting to understand.
You know, I- we call it internally
the Claude wall, uh, but it
could be my chat tool wall.
It could be Copilot.
It could be ChatGPT.
It could be, you know, uh, devs.ai.
It could be HATS.
It could be anybody.
When we hit that personal productivity
wall that you alluded to, it, you just,
you can't get anything else out of it,
and now your two options are, do I go the
old way and build an app, and I manage
that app and the infrastructure, or do I
figure out how to use the modern stuff?
And that's, that's where we come
in and some of the others come
in to deliver agents, uh, without
having to host the infrastructure.
W- build MCPs without
having to spin something up.
We had a partner the other day that
said that one of their clients connected
their banking software through our
Excel app, uh, using a, a, an MCP
they found on GitHub because Claude
told them to, to, to go do that.
And it had two stars and no comments,
Todd Kane: This sounds scary as hell.
Callen Sapien CEO Synthreo:
banking through
Todd Kane: Yeah
Callen Sapien CEO Synthreo: And so, like,
those are areas where governance makes
sense, and they still need to do that.
So build the MCP.
Uh, offer it.
Even if you don't use 3.0, that's,
that's… For me, when we build
an MCP, it's universal, right?
If someone is, is dedicated, if they're a
maxer and they're using Ultra and they're
going, you know, in Groq, and they're,
you know, they're taking their toket subs-
token subsidies to the next level, don't
necessarily want them running on my, my,
my, on, on that area, but we still need
to give MSPs a way of managing that.
So that's when we, we build MCPs,
governance policies, observability.
And I think all of the things that we've,
we managed before that I hope people
don't take this pejoratively or, or,
or as a wagging finger, but that we've
kind of went away from over the last
five or six years as we've resold a lot
of SaaS and a lot of security products
without wrapping services around it,
uh, is, uh, is it needs to be relearned.
And one thing that's a little scary
to me is we've had so much M&A in
the last 10 years that I wonder
how many, like, experienced service
delivery people exist out there.
You mentioned John Da- Dobbin.
I, like, that guy's
worth his weight in gold.
He has 24 years of delivering service.
If I was an MSP, I would be grabbing onto
him and saying, "Okay, should we sell AI,
and how should we build that offering?"
And find that service manager that is so
passionate and had been in that thing,
and have them reteach the entire company
on how to deliver services at scale,
and then base your AI policy on that.
That's, it's get back to the basics,
which sounds silly, but it's,
it, it's, it, uh, the, the idioms
are, are there for a reason, you
Todd Kane: Yeah, there's
so much truth in that.
Like, I find everything in business and
somewhat, uh, I guess a large degree in
life is you, you getting back to basics.
It's just we've gotten so far
away from the simple things.
Like, I always tie it back to, uh,
you know, the John Wooden a- approach,
very famous basketball coaches.
We're gonna start with tying our shoes,
and everyone's like, "What the hell
are we doing?" It's like, eh, you know,
tie your shoes correctly, you don't
get blisters, and then you're not out
of the game, and then a higher point
scorer s- is in the game, we win, right?
Like, all of this stuff from,
starts from absolute basics.
So I think there's a lot of
Callen Sapien CEO Synthreo: Угу
Todd Kane: truth in that.
Uh, and I am right behind you on, uh, you
know, let's get passionate about this.
Let's start building an AI
practice because, as I said, your
clients are doing this without
you, and they're doing it poorly.
So they absolutely need your help.
Whether or not they've explicitly
asked you or not, they need your help.
So no, this has been great.
Uh, really appreciate
you coming on, Callan.
Um, if, uh, people wanna reach out
to you and know a bit more about
what you guys are building in the
platform, where should they find you?
Callen Sapien CEO Synthreo: Uh, either
Synthrio, uh, .ai or you can email
me directly at callans@synthrio.ai,
and I'm happy to, happy to jump on.
I've got a, I've got im- I've got-- My
link is public, so I love this industry,
and thank you so much for having me
because this is a mission for us, right?
We, we-- When we were building this
company, we, we saw this problem
coming, and we actually thought,
"What if we were to build…"
Because my co-founder built a
successful MSP and sold it and exited.
He's like, "What if we built an
AI-native MSP and just went and sold?"
And w- both of us realized how much
that this community has given us.
I don't care.
When I say community, I mean all
of it, ConnectWise, Autotask,
Halo, all the, the individual ones.
And we, we just want to help
get back to those basics.
And, and so yeah, please reach out to
me, even if you're not a partner or
interested in being a partner right now.
W-w-we want to help
Todd Kane: Cool.
So I'll link to, uh, everything in the
show notes, but, uh, it's been awesome.
Thanks, Calum
Callen Sapien CEO Synthreo:
Thank you very much