Subscribe
Share
Share
Embed
Today I'm speaking to Mike Psenka, CEO of Moovila, once again. If you missed our previous episode together, Advanced Project Management—episode 104—I highly recommend checking it out for some great insights.
Today, we're diving into a topic that's been top of mind for me and, honestly, I believe not enough people in the MSP industry are paying close attention to it: project management and profitability. We'll break down some eye-opening stats about how 25% of the MSP channel is either breaking even or losing money—and how poor project management is often the hidden culprit, even when service delivery is profitable.
We're going deep into why projects are causing financial headaches and customer churn, and, more importantly, what MSPs can actually do to fix it. Mike and I will talk through real-world challenges, practical fundamentals, and how AI and automation are changing the game for project delivery.
Whether you're struggling with projects, worried about client retention, or just looking to level up your operational maturity, this episode is packed with actionable advice.
This episode is brought to you by Opsleader Pro. A place for MSP owners and managers to get the systems and tools they need to build a stable and growing MSP. Part group coaching, part peer group, everything you need to run a successful MSP.
Evolved Radio Podcast: Interviews with technology experts, industry thought leaders, business leaders and other interesting minds. Exploring the evolution of business and technology.
Mike, welcome to the Evolved Radio podcast. Thanks for having me,
Todd, great to. Have you back as well. A returning guest.
I think your previous episode, Advanced Project Management, it
was episode 104. If people want to go check out
some of the fun stuff that we talked about there.
I want to start us off with basically
my favorite stat that I've been quoting kind of the last two years,
two stats actually, because I still feel like
this is something that people are not alarmed enough about. So I will continue
to talk about this. So the first stat, one of my favorite stats in the
industry, people have probably heard this because Paul Dipple has been talking about this for
a long time, is that a full 25% of the industry
and the MSP channel is break even or losing money.
So alarming stat enough to begin with, right? Everyone's kind of
heard this probably at some point. The really interesting part of this
was service leadership dug a little deeper and in the data they
found that in that 25% that is struggling,
most of them actually have profitable services. The problem is,
is they're so bad at project management, they're cannibalizing all
of the profits from the other departments of the business.
And when I saw this, I was like, this absolutely makes sense
like this. Why are people not recognizing this?
They continue to sort of fight through service and automation.
Projects still tends to be a bit of an afterthought in a lot of
immature MSPs. And I don't think people recognize the
damage that it's doing. And you guys obviously have a bit of a front row
seat to this. I'm curious, sort of your take on, you know,
how true you find this to be and what do we do about this? Both
in promoting the idea of projects is how you fix your
MSP profitability and then how do you go about that, I suppose.
Yeah, yeah, absolutely. And it's a great question. And we know that statistic well, see
that and kind of advise organizations around that. I guess there are a couple
of components and this can kind of be split into several areas of sort of
why does this happen for businesses? One of them is organizational
maturity, right? A lot of MSPs as you know, even self
describe, are accidental entrepreneurs. The way their business grows up and what
they're doing. And so this idea of sort of disciplined project
management around that process wasn't necessary if they were a
single engineer or growing from small process and kind of moving forward because,
you know, the human brain is amazing at managing a certain number
of tasks, process and sort of Quote, unquote, the critical path in your head. They
know the things to do to execute process, and when they're doing it on their
own, there's no repercussions. But there's a point that, that business
scales to a certain level where that no longer can be
maintained in someone's head, how they're tracking and managing
that. And so that becomes an issue organizational maturity. But
this still happens even in a larger business
where organizations struggle because of the scale of what they're
doing and how they're tracking and managing all of their
data. And so one of the big issues that we talk to
organizations about is as they go forward on this is
managing, how are you managing the critical path, your projects and how are you structuring
your data? And for most organizations, they're just,
I guess there are a couple of things we'll talk about structurally, how they manage
projects. So it's sort of, it's sort of like you're saying,
why is this program running badly? Well, because the code is bad, it's
got lots of bugs in it. Would you ever expect software to run well that
had buggy code in it and incomplete code? You're like,
no, that's an absurd notion. But running projects that
have really buggy plans that weren't complete and so
it's aren't predictable, they aren't accurate, they're managing that. And I don't think
that for a lot of these organizations. One, the technology
that exists in the PSAs that they use don't
have super robust project management platforms. And
so they're managing shared lists. And lots of times
when they use third party tools, certainly non integrated third party
tools, they're using like shared lists that aren't
reconciling with their psa. And this can become a really dangerous,
risky thing. And so once again, the surprise is
why are these projects going badly? Well, then there's
the next thing, right? So let's say you had, even if
you were managing the data in a third party system, to
overcome the limitations of your PSA and
you had really capable PNs in there.
The thing that nearly everybody doesn't realize is
that statistically for your project you're
going to have problems. So the statistic that we'll often quote
is a project plan that has 40 tasks on the critical path. You
know, the dependent chain of tasks. For those listening who are not familiar with it,
the critical path is, you know, this task is dependent on this, is dependent on
this. It's that longest chain that defines the
minimum length of that project and there could be
100 or 200 tasks in the project. But if there were 40 on the critical
path, if everybody got their
work done on time, 90% of the time, that includes the
customers and their responsibilities and distribution.
Let's be real. And I think we all agree that's a ridiculously
high number.
Sorry, Mike, I cut you off like, oh, can you restate that? Yeah, yeah, I
was just saying so. So I think we can agree that's a ridiculously high. Like
I don't think any MSP would go, yeah, my engineers and everybody gets their stuff
done on average 90% of the time. That's a high
number. Self reporting in. But even if they did get it
done, there's only a one and a half percent chance that that project would be
on time. Even if it was beautifully built, beautifully
structured, with all the detail in the world, the odds are horrible.
And these are the same odds. To put it in the context that
msps can understand, a task in a project
is like an endpoint you're managing. It has a
failure rate. And by the way, that task has a much
higher failure rate than any endpoint you're managing. It just
does. That failure is, oh, the parts didn't come in. I didn't know I was
supposed to do it. I had to go to the customer site and it got
rescheduled. So once again, you think about the ecosystem
that you're managing. You have a lot
of endpoints out there, a lot of tasks. And for some MSPs, they have as
many tasks and projects as they do endpoints that they're managing over the course of
a year. Right. But they get surprised and they think that everything's going
to go off perfectly and the failure rate's higher. So they're like, you're telling an
MSP, what are the odds you're going to go a couple of months with not
a single endpoint problem? They would laugh at you. But
they don't realize the failure rate is so much higher on the task. So that's
the other part of this is that the, I'd say not just MSPs, the whole
world doesn't realize all projects statistically are doomed to have problems.
Now, what are you going to do about it? So hang on, let's pause on
this because this sits on one of my other sort of favorite stats of
project management in IT is that
65% on average IT
related projects are a failure. Right? Which is a pretty
shockingly high number based on what you're saying around like the
failure rate of projects. Do you think that that's more global across projects in general?
Like projects as a whole are fail. Are tend to be failures or is
the inherent complexity of it projects that leads to a higher
failure rate? Yep. So, and I'm going to differentiate
between delay and failure. Right. So there are lots of
projects that are delayed, that are ultimately successful, successful
projects, but they were delayed. So when I say there's a one and a half
percent chance that that project will be done on time as planned.
So you've got to realize we're going in this, we're going to have delays. Now
if there's a drop dead date, you better know that you're going to be
scrambling and you know the term crashing that project because
you're likely to have problems unless you're at the 99
percentile or 99. Like the probability is just bad. So,
so that other statistic gets confounded
by the fact that there are delays and risks and those projects get
killed because confidence is lost. One of the big things that we
always talk to a lot of our customers about and I think one of the
interesting things that we learn
improving to the point of the benefit to
improving project management is not only the margins, which is
really, really important atomically when you look at a project and go, hey,
we were profitable on this project. But the other really
super important thing that we learned over the last couple of years that We've heard
from CEOs of a lot of our customers was the fact of I
have better credibility now with my customers, which means they're a better reference and they're
more likely to renew. In the past, when I was missing dates
which they didn't know they were doomed to have problems with and they didn't
know when they were missing the dates the customer would call him and go,
feel like you should have known about this, feel like you should have known about
this delay. And by the way, if, if you didn't know we were going to
be a month late on this, what else are you missing? And a
lot of these organizations lost customers and big customers because they were.
The customer went, you're making me stressed because
you don't seem like you're on top of this. And they're, they were totally on
top of their managed services. But it gave the
appearance of. So there's a, like to your point, there's
two really important reasons why do you want to get your house in order on
this. One, don't lose
money. You don't need to lose. But Two, this is your reputation and you
know, we're an AI vendor and AI project management, blah blah, blah, and automation.
But I don't care how much AI and automation comes into the
MSP industry will be relationship based. I think it will
always be relationship based. And if it isn't relationship based, it
goes, goes away. I mean, so that
relationship is based on credibility. And one of the, what I would
love to hear, and I don't know how we get to this statistic is how
much credibility damage is occurring. You know, you're saying, oh, 25%
lose money and that's based on bad project management. What
percentage of churn is related to the fact that they dropped the ball
in a project? Not, not race to the bottom on pricing
per node, not other things, but damaged
customer sentiment because hey, you mismanaged some
really important projects to me was really frustrating to me and my team
and you know, maybe if you'd communicated better,
we wouldn't be so mad. And that was the other issue. We'll tell clients
you're not going to eliminate delays. This
idea that you think you're going to ever prevent a delay with the world's best
project, but you want to know early, you want to know soon, so you can
call your customer and go, hey, we're going to have a problem in six weeks.
Not oh, we had a problem a week ago and it's going to create a
six week delay. Like that's what happens for a lot of these organizations. And
so like I said, there's a two hit thing. You lose money and then you
lose customers and you lose reference of all customers with bad project management.
So yeah, this is, this is huge because like this is, I'm
pulling out all the stats today. Like this is the other thing that I found
really fascinating, especially kind of two years ago. Similarly related,
like I was doing this talk all year about sort of efficient project management.
So I was talking about these things ad nauseam all year.
The other piece was that more MSPs are losing customers
due to project failures than service failures. And that's a switch. Like that's not
always been the case. It was usually service desks that caused that loss
of trust. Of like I submitted a ticket two weeks ago,
no one ever even called me back. Right. Like that stuff is what eroded
that trust. Most people, I think of most organizations have gotten
to a place where their help desk is pretty good. You know, there's obviously
like a spectrum of people's capability of maturity on delivery on that side,
but projects again, like it's just, it's really dropping the ball. And I've heard multiple
stories from MSPS where they got completely blindsided by
customer customers canceling their contract because of
project failures. And they were, they were, they were totally caught off guard.
So I think again, like that's a really important reflection of like this means a
lot because now, now not only you more, more likely to
lose money because of projects, you're actually more likely to be fired because of
projects and lose customers as a result. Yeah, it's a double hit.
Yeah, it's a double hit. All right,
so I want to understand a bit about what we do about
this, right. Because I think there is a general baseline like,
like what you're talking about, I strongly believe in is that there are just
fundamentals that people are not putting in place in managing
projects. Right. Like a lot of this just happens off the side of someone's.
Because you know, the tyranny of now, you know, I do this whole band,
busy campaign and everyone excuses the fact that
the stuff doesn't get done because I'm busy. Right. And
that because of these reasons, that's unacceptable. So like where do you
start? Right. I guess is the important aspect of this because
sure, like I'm always, I would say I'm the first to admit the
psa. Most of the project modules in there are okay at best they're not
great but as you say, don't go into a third party tool
because that's not the problem. It's the
fundamental systems that you're putting in place, the expectations of how
these projects get scoped, implemented and managed. And that doesn't
require a lot of rocket science. Like that's just the basic tasks of this stuff.
And then you can layer on the complexity on top of that and make things,
you know, spin tops and do amazing things and automate
stuff and. But you need those fundamentals in place, right?
Yeah, yeah. And I guess, I guess a couple of things in that regard and
just at least from, and everyone has different sort of industry perspectives and
I, and I hear what you're saying as far as, hey, what is the need?
What is the baseline? It's, there's not one quick
solve for it. And it depends on the size of the organization.
Right. And what are their problems and their challenges. The
larger organizations will say for sure it is a technology problem. The
PSAs are inadequate to do that and manage the scale when you know,
we have customers with 3,500 projects
in their portfolio with projects of a thousand tasks,
you know, individual Projects that have a thousand tasks in them, the
PSAs cannot manage this in any way, shape or form and it becomes a
dumpster fire. Now, I will say using a third party tool that is not integrated,
you're. The costs and the chaos of that are a huge problem.
Right? So having a third party with swivel. Chairing between two systems, you
have no data integration, right? But like layering something on top of the
psa, I'm still a fan of that, to be clear. Yep, yeah,
yeah, we have customers that come in and go, oh, I don't want to do
the integration. We're like, yeah, you do, you do want to do the integration. Because
you're just not going to want to deal with that. And, and having the integration
where it's not just the tasks and its completion, but the structure,
the product process, the tickets, the time, the notes,
all of that, the custom status is the role, all of that stuff
has to go back and forth because you want your engineers looking at the PSA
is a source of truth for their work, right? You have your PMs and
you got engineers floating in between regular ticket
work and project work. So, so having a place where
accurate data can live is critically important. Once again, I'm not
saying anything that. And I will say the PSA vendors and the heads of products
have come to me and talked to me about this, going, hey, we know this
isn't kind of great in our product, but that goes into resource management
isn't just the project management because what's important to the MSPS dates isn't
just about the project delivery dates, it's about effective resource management because
that impacts the margins, right? So, yeah, automating
scheduling. So I'll say a couple of things, right?
People do things that they have to do and want to do. They don't do
things they should do. So you and I can sit here all day long and
tell people, you should do this, you should stop smoking or
stop drinking or lose weight. And nobody will do that
unless they want to or they have to do it. So
the first thing that any MSP listening to this has to
look at it and go, like, if they're listening to you and I and they
see us as pontificating about best practice and what you should be
doing is not going to do it. Don't waste your time. But if they go,
hey man, I am tired of this, like, I have to move to the next
level. This is painful, I'm tired of the embarrassment. I don't want to waste money
unnecessarily there's a real want there. And there's also a risk
for them to sort of say, hey, we have to do this. And you know,
if someone's not willing to kind of go, hey, we have to do this, then
I do believe anything we say is a waste of time and they should just
turn this off. Sorry, I don't mean to like reduce the viewership or even
watch the rest of it, but if they are willing to do it,
there are two things that we always say, hey, maybe you're not
doing this today and you have to add these two data items that
maybe you're not doing today. Like, so they're describing what the task is
today, right? Hey, we've got to go install a firewall or
we have to do this or we have to do that. And they're assigning a
resource and maybe they're putting a work estimate in there.
Like we think this is going to take three hours, that's our budget, four hours.
And they know their cost. But two things a lot of them aren't
doing. We say the double D is a dependency and a duration.
You know what has to happen before this? Sometimes there
are different types of dependencies, but they don't even have to go down that path.
They have to give it a duration. Like, hey, I want you to
configure this firewall. I understand it's going to take an hour, two hours, whatever it
is, I'm going to give you five days to do it. That
dependency and duration, the minute you do that in any project plan,
you've now programmed work that unlocks a
ton of automation. It also means your work can be debugged. It
means you can autonomously look at resource capacity conflicts.
You can automatically schedule. Like it unlocks a
universe of efficiency. But if you go, I can't
be bothered with that. My team, they don't do it today, they're going to find
it to be a hassle. Then you go, okay, but just so
you know, you are
abdicating your ability to ever get out of
this hell of the unpleasant surprise of why were we late on that?
What happened with that project? Why did we lose money on it? You're
foregoing automation. And it's almost like you're saying we want to be a break fix
business. No, MSP would go, we love being a break fix business. And people
who aren't automating project management are running break fix projects.
That's what they're doing. And so if they're willing to add those
two Other details, hey, dependencies and durations. If the
technology can support it, they now have the ability to
automate a lot of functionality, accuracy and
predictive capability in the platform. So we always say like,
that's a, for us, that's a pre qualifier. We're like, look, if you're not willing
to do these two things, you're just not going to reap
benefits that AI and automation can provide you in project management and you're
just going to continue to get blindsided. So you have to do that math, figure
out whether or not that's worth it to you. Well, this is the part, like,
one of the things that absolutely drives me crazy about the way that we do
projects in the MSP industry because we found this business
model that we get paid regardless of whether or not we're doing work, which
allows scale, right? Like the MSA model versus time and
material. That's why margins are 60 to 70% instead
of 25. And then we pivot to doing projects and
go right back to the TNM model where we're just paying for hours, right? Like,
how many hours did we do? What are we billing for? What's the labor rate
on this person? And therefore how much margin can we make? The only way you
make more money is by adding more people and more hours. So you're like,
why have we given up on the MSP
model for projects when we recognize how great it is for our
businesses, right? I think like this, this drives me nuts of like not
being able to plan, resource plan and understand sort of how this project
gets implemented so that it is, it is more predictable. I think
it's a call to arms for people of like, work towards this. This is not
something you can do tomorrow. But these are the basic things that you need to
start to put in, can evolve your process and make
it more mature over time so that you can move from 25%
to, you know, 40, 50, 60% projects.
Well, and this is one of the things that also, you
know, we talk about limitations of the tech that they may be using in
the PSAs is this ability to analyze the performance of their
projects. Like, so if you're going to go to a fixed fee model and we
talk about this, you're going to do a fixed fee model, you're not going to
have the time to do a retro. Like large organizations, when they have large monolithic
projects, they do a retrospective and go, how did we perform? What did we do?
What do we do? But msps, even the large ones, don't have the Bandwidth to
do these retros to understand. Where did we misquote this? Where do we need to
modify this? So once again, coming back to
listing dependencies, durations and work estimates and being able to
automatically analyze that template that you're using and
this idea of productizing services to your point, right? So, hey,
what is our service delivery offering? Let's create a template.
Let's make that a product that we're going to offer. We're going to estimate the
work hours. But then being able to understand after the execution of that project, hey,
we got it wrong. We thought that was going to be two hours. It's actually
six hours. Okay. When we bid this out again, we're going to bid it at
six hours and, and knowing that ahead of time and because they're
not doing retros, to your point, this is one of the areas why
so many of these MSPs lose money in their projects. They're scoping because they have
no ability. They're not big enough. They don't have to. Even the big companies can't
do the appropriate analytics to dial that template in. And the
other component is scope creep, that they don't have a structured
process to capture new cost and then move
that into the customer and, and you can't tell a customer. And
this is one of our favorite statistics too, that we talk about is
they'll identify cost overruns 60 to 90 days
after when, you know, if they don't have an automated way to
do that retro or identify the risk automatically. Once again, this is the benefit of
AI and automation during project execution is you identify when
it's coming off the rail. So, you know, if I was renovating your bathroom, Todd,
and I came to you midway through and said,
hey, you have rotted floorboards. We pulled the tile up. I know you want a
new tile. You got rotted floorboards. We're going to have to rip that out and
put it in, blah, blah, blah, blah, blah. And it's going to be an extra,
you know, six grand. Like, but we kind of need to do this. You're gonna
go, yep. Okay, now imagine, finish
the bathroom. I bill you, and 90 days later, after
billing you $38,000 to redo your. Bathroom, by the way,
I replaced your. Four, by the way, we had to rip out the
floor. I forgot to tell you, we had to rip out the wooden floor to
do this. It's another six grand. You're going to go, no,
like you were willing to pay it in the middle of the project, but you're
Now, a hard no, because you paid that bill 30 or 60 days
ago. And so this is another major reason
MSPs get stung on this. And once again, if they're
willing to make some process changes, and you preach this a lot, and I know
you're trying to develop and elevate their experience for project management, but there
are big dividends that they're willing to, you know,
one, change their process a little and get the right tech in place
so they can live real time, identify these risks, the
delays, real time, the budget overruns, real time, because then they can
adjust in flight. And once again, and I'll shut up here in a
second, it goes back to that endpoint model. Like, do you want to find
out about that, that noisy node or potential problem
early or do you want to find out after it blew up and took the
whole network down? And right now, once again, for most MSPs that
don't have discipline around this, it's blowing up and taking the
whole network down, I. E. The whole project down, that they don't have any kind
of early detection or warning about that. And this is like, I think,
like, this is full circle, right? Because I feel like the
part of the reason that people are not able to manage those costs on
the fly is that they're uncomfortable having these conversations with the
customer around a change of scope and cost. Right? But they're.
So, they're just absorbing it because they're like, well, you know, the customer will probably
say no, maybe they're going to be angry at us. They, they sort of justify
the idea that they should eat the costs on the project or the scope
change or the overruns without sort of having a transparent business
conversation about that. The problem is, is I think most people don't recognize the
impact of that, right? Because their P and L has a
blended service between help desk and projects,
right? So it's like, well, you know, we're, we're doing okay not
recognizing your Service delivery is
60%, but your project delivery is like 5 or negative
15, right? Yes. And
the thing is, some MSPs are like, well, we're just not going to do project
work anymore. And you're like, okay, so now you're opening the door for a competitor
to come in and do that project well and take your business.
So they're like, this idea that you can ignore this problem and
it'll go away or just give it to somebody else is
chording churn, right? Yep. All
right, so one other bit I wanted to get, get your, your Input on
is how much detail is enough. And maybe this is kind of a continuation
of what we're talking about here because there is sort of this happy
medium somewhere between like it just
a gory amount of detail and everything has checkboxes and
you know, every little, little unit is tracked. You know,
been working with this organization. They do a lot of physical deployments and it
has a lot of
individual devices in numerous places. So there's a huge amount of
information to track here. And they're looking to improve their project delivery
process from where they were and they're getting good gains as a result of that.
But they're getting a bit of pushback from or from the staff of just like
oh my God, like this spreadsheet is eight pages long
of things that we kind of need to go through and verify in order to
make sure the project is complete. And like I'm of two
minds of this recognizing like look, that information is important. This is just
the stuff we need to do on the project. But if you're coming from a
low maturity state and you're asking for high maturity delivery,
then there's going to be a lot of friction in sort of how you move
that ball forward and get the buy in from the team. Any thoughts on sort
of a circumstance like that of like we're meeting resistance on trying to
do the right thing because maybe we're asking for too much too early.
Yeah. So the ROI
is sort of a step function on this about the changes an organization's
willing to make and the benefits that they're going to receive. But and there's a
little bit of this. Don't. If you're not going to do at least this much
then don't bother doing anything because it's going to be a bigger problem. So
let's talk about detail in two different
structures. There's the detail of number
of tasks and then there's the detail within
an atomic task. Those are sort of two levels of detail that you have
to contemplate in this. So one of the biggest mistakes that we see
MSPS make and we actually are head of operations
who speaks a lot in the industry has a. I think he's got a webinar
on it called the Project which is basically the entire project is one
ticket. Like they're so used entering tickets and they just load the whole. This
is clearly an insufficient level of detail for a large scale project and
you're once again courting disaster by loading everything into
a ticket. So there's the issue of what is the
appropriate level of detail and you could do a whole
session on what is the level. And people now could use
ChatGPT to find out the appropriate level of detail for the number of tasks on
stuff. So I don't need to cover that. But, but, but certainly more than
one for a large project. And that's going to vary on the scale of the
project, whether it's 20, 30, 50 or without. Like I said, we have, you know,
customers that have over a thousand tasks, MSPs, you know, that do that for
their customers. The second thing, so, so
look and get some level of detail. But the second thing is what do you
have to do on an individual task to execute function?
And when we talk about sort of bare minimum, you need
to know what the task is. I don't think anyone would debate, like you have
to describe what that task is and have detail
associated with that as a bare minimum, you have to
assign that to a person. Like, you gotta do
that, you gotta assign that to a person. And then for msps, because of their
financial models and their budgeting, you have to put a work estimate on it. We
think this atomic task configure
this Device typically takes 22 minutes or
typically takes an hour and a half. Whatever it is, you got to do those
three things. And then to unlock
automation and accuracy, you gotta have dependencies and
durations. Like you gotta have those because if you don't,
then you're gonna be in the blind, you're gonna be kind of blind again. You're,
it's gonna be difficult. Now there's a lot of additional detail that you can add
above and beyond that, right? A lot. But when, you know,
when we try to coach organizations and go, hey, we know today,
you know what the task is, you've already got that. You're doing that in your
PSA and saying like, we got to do this. And you kind of know that
list of things you should do. You know
the resource type at least that you want to assign it to, if not the
person, you know the resource type that you're going to assign it to and you
roughly know how long it should take and if you get it wrong,
you can update that. But once
again, if you're willing to say, I mean, how long do we think we'll give
someone to do this? Three days, four days, and what is it dependent on?
You add those two things, you can unlock, once again, automated
timelines process and updating, automating your
timelines, because it's going to change statistically. You know, we talked about
you're going to have problems and that means the project's got to be recast. And
once again you want to be integrated with the psa because the minute you recast
that, you want all your engineers tasks to get their
dates updated automatically to go, hey, this is the new
normal. Just quickly on this. This again
calls forward. One of my favorite quotes on projects from Patton
is planning is everything and plans are nothing. Yeah. Because
the assumption is this is going to go sideways so you're going to have to
replan as you're going. Right. I think to this, like don't meet, don't
recognize this as resistance or a failure. This is, this is just how things
go. Like you've got to be probability. That's the point. We, you know, come full
circle. Like you said in the very beginning of the conversation, that's probability. So but
this is what you want. You want AI and automation
to recast that reintegrated and updated. You don't want.
And this is why project plans are terrible because when people deal to try
to manually do this, it's not possible. And so they get
garbage dates and they're communicating garbage dates to their customers. So
this is why, you know, what we're passionate about is to say, look,
we statistically know that this is 100% chance your
99.9999 chance you are going to have
a problem. We talk about the portfolio of work. It is a
septillion times more likely that the sun
won't rise tomorrow morning then your project
portfolio, if you have 20 of these 40 task projects is going to go off
as planned. Like the odds are terrible. So what are you going to do? And
that's where we step up. Like what do you plan to do about this? Hope
that it doesn't happen. I mean, good luck with that.
Yeah, let us know how your custom. Right. All right, so
and this I think gets to, I think a really interesting space in this again
like PM in general being somewhat
underserved in our industry. And this is sort of an interesting
segue because a big part of what you guys do and bring to
MSPS is a level of AI and automation that is
absolutely not available in other platforms. And I think it's
the stuff that you guys add is super cool, especially because as you said, it's
integrated, it's not a standalone platform. So the fact that you're, you're servicing the
MSPS in this way is, is really interesting. That said, you
know, we talked about this two years ago on, on a previous
Episode two years ago in AI is a frigging lifetime.
So I would be curious on what has changed with sort of the
base capability of those platforms and you know, the back end
mechanics of what you guys can bring to bear on projects. It must be somewhat
radically different, I assume. Yeah, so, so I'll
say this. So. So we rolled our own over a very
long process using deterministic, symbolic
and heuristic technologies. The new so
and we also have other things where we use the latest gen AA stuff in
hybrid fashion. Right. And so there are a couple of things that are really important
about to know about this. So in our platform
because it's deterministic in nature, it means it's repeatable,
interpretable the results and why we got what we got. I think everybody
now dealing with AI knows the idea of hallucination in the generative
platforms and the probabilistic systems and you don't know why it's
giving you the wrong answer. The thing that, that we know
now is true because like, you know, obviously we went and certainly when
ChatGPT came out we went oh gosh, is this going to be an existential risk
to what we're doing? Are people just going to be able to throw garbage project
plans or any project plan into an LLM and it's going to just tell
you how to do everything and fix it? Right. And so we did research
and fortunately through connections have access to people who are in research
circles around this papers as recently
as this year coming out of Harvard, Michigan State, Microsoft and
Amazon, all of the LLMs. And this
linearly probabilistic generative AI solutions are terrible with
graph data. So and what I want to, I just want to sort of unpack
when we talk about error rates around a solution.
So for scheduling a project like hey, I
want to schedule, we talk about what's an acceptable error rate. We have different error
rates for that we as a consumer will accept
around the task of automations and those can get expressed
of acceptability of percentage rates of like a less than a
half percent, 10%, 5%. You know, if you're
writing a poem for one of your kids or whatever, like an error rate of
not quite even getting. You don't care about, about that. Right. But if you're saying
what's the error rate of my parachute opening? You're going to want that to be
less than, you know, 100th of a percent or a thousandth of a percent chance
that your parachute and that might even be high for the number of people go
Skydiving, less than a half a percent would be
range or quarter. What would be expected for automating
project scheduling? Like getting that out and saying, oh, this is when this project's going
to be due. This is what, here's your scheduling problem,
here's your conflict, here's your resource issue. Right
now, the error rates for teeny tiny graphs and
projects, but is 40%. It is so
extraordinarily high as to not even consider and there's
no improvement. So these models, if you look at the
drop in error rates for the models on on language
in 2D and 3D vision, those error rates drop
significantly because of the nature of what they're doing
generatively. So I'm generating text and a
letter for you, or a business plan, or I'm generating an image.
That generative process, they've really been able to drop it.
But the idea of observing a living dynamic system that is in a graph,
it gets lost. It can't relate beyond its nearest neighborhood,
it gets confused. And because it can't traverse a tree
in a depth first search or breadth first search,
it falls apart and gives bad data. And unfortunately,
like some of the things we talked about before, you get one bad answer in
that graph and that cascades because it's all connected.
So, you know, we look at this and go,
any of you hoping that you're just going to be able to plug your crappy
management into an LLM and get the solve? And you're just waiting for
that because it's AI and it gets better.
There's no line of sight of that and there's no papers. So ChatGPT
came out and it was amazing. I think 22 is when we started to go,
wow, that's really something. I think that paper came out in 18.
And certainly things move quickly and we can say things happen overnight,
but there was line of sight. And we did know back when that paper
came out that, oh my God, we can parallelize this computer.
We just need to get more compute and more data. They, they weren't
surprised when they were going down that path. They knew they were going to get
better results. They knew those error rates were going to drop, which is why all
that money flowed into it and why there was a hype cycle. And we as
consumers saw the benefit and we're like, oh my gosh, this is
amazing. Once that error rate dropped to a point where we trusted it once
again, they're nowhere near that for managing projects. And
so we look at this and go, you got
to use traditional you need to have an expert system in place, deterministic. You can
layer, we have customers that layer other AI stuff on top of our data because
they're operating off of a foundation of accurate data. Like they're taking the
guesswork and the hallucination work out of it. And then they're going, okay,
summarize this. Yeah, this is interesting because, like, I play with
enough AI that I recognize what you're talking about in the boundaries of this. Right?
So I think a lot of people saw
early on hallucinations where you would ask an AI about something and it would
just completely falsify something. You can tell it's, it's, it's not
even remotely true. My brother had a great example of this. He asked
GPT a long time ago, like, who was, who is the, who ran
his company, and it responded with a real person that had
nothing to do with his company. I'm not the CEO of my own company.
Interesting. Okay. But that was the early
versions of this. And it seemed like a lot of that got solved because
hallucinations dropped precipitously. But I
still regularly see situations where, like, it has no idea what
it's talking about, even in basic math, right to the point where it is trying
to generate the next token. And unless it's, you're asking it to solve a
particular math problem, it can get some of this stuff absolutely wrong, which is
weird. And the other part that I see this show up and is, and probably
relates a lot to projects is I am
regularly fascinated by the fact that GPT has no idea what day it
is even when I tell it, you know, and then I'll, I'll say the data
is this, blah, blah, blah. And then we'll go through like not a huge context
window and I'll, I'll try to ask it to plan for something else. And it
completely misses the mark. Like it's like it's April 2023. You're like,
what? No, where are you? Right, yeah, well, and you just
hit on another really key point of why it's limited in projects.
Right. Is this the larger. The context window, that
performance rate decays in a non linear fashion, so it
can perform really well with a paragraph or two of data. Goes off a cliff.
Any large project plan blows up context windows. And
so like I said, it's just, there's no line of sight that, that's gonna, you
know, I think this idea of, oh, they'll figure it out, they'll figure it out.
You know, like the figuring out is A hybrid solution. Like you're. And
it is doable in a hybrid solution today, right? I mean that's,
that's the big question. But you're spot on in saying like,
and projects have a lot of data right? There is at the end of the
day, a lot of data. When you're talking about what are the engineers on. Even
on a smaller project that has 15 tests, well, who's working it, what are their
schedules and what are their resources and what are their skills and what's the time
like? Once you get to that level, it blows up.
It might be able to generate a plan, but it can't observe it, monitor it,
identify risks and problems. And, and that's,
that's the issue right now that these MSPs face is, as you said, your
patent quote, like, oh, we had it to start. Why did it blow up? Well,
it was doomed to have problems. You didn't know that because humans suck
at probability. But it was doomed to have problems.
You just didn't put an early warning system in place through best practice,
process structure and technology so that when that inevitably
happened, you could adapt to it and adjust. You're doing that with your
endpoints. You pivoted from being a break fixed MSP
to a managed service provider that's actually observing through your
noc. You got to kind of build a NOC for your projects.
Yeah, yeah. I mean, I guess that's the project, right? Start,
start working on, on these iterative improvements, get the basic
mechanics in place, start working on resourcing dependencies, time
windows, and then start to pick up steam from there.
So really appreciate this, Mike. Thank you so much. If people
want to get in touch with you or your team, what's the best place for
them to check.com M. O O V as in
Victor I L A dot com. Yeah. And we'll be at, we go to
a lot of shows during the course of the year, so hope to see you,
Todd, or other folks if possible, at those shows. And thanks
everyone for taking some time to talk about projects because I know both
you and I are interested. Yeah, yeah, we're pretty passionate about it, but,
but glad for anyone who's also expressing interest and happy to answer any
additional questions. Awesome. Appreciate it, Mike. Thanks so much.