00:00:00 Dr Genevieve Hayes
Hello and welcome to value driven data science brought to you by Genevieve Hayes Consulting. I'm your host doctor, Genevieve Hayes, and today I'm joined by Max Stouse to discuss how data science can be used to comprehend the underlying cause and effect relationships in business data. Mark is the CEO of Proof analytics.
00:00:23 Dr Genevieve Hayes
An AI driven marketing analytics platform prior to becoming an analytics software CEO.
00:00:29 Dr Genevieve Hayes
Mark had a successful career in B2B marketing and in 2014 was named innovator of the Year at the Holmes report into Sabre awards for his work in tying marketing and communication investment to key business performance metrics. Mark, welcome to the show.
00:00:49 Mark Stouse
Hey, it's a pleasure to be with you.
00:00:51 Dr Genevieve Hayes
Correlation does not equal causation, as anyone who has studied statistics or data science.
00:00:57 Dr Genevieve Hayes
No. As data scientists, we are trained to consider this when developing predictive models. But understanding cause and effect isn't just important when you're developing models. If you're working in business and want to be recognised for your work, it's essential that you will be able to demonstrate causality.
00:01:17 Dr Genevieve Hayes
Between what you do and the benefit flowing through to the business, and this is something that you mark have managed to achieve in the context of marketing and communications through proof analytics.
00:01:33 Dr Genevieve Hayes
So to begin with, can you tell us a bit about what it is that proof analytics does and how it came to be?
00:01:41 Mark Stouse
Sure. So about, I don't know, almost 20 years ago I was at HP working for a guy named Mark Hurd, who was the CEO at the time. And at that moment, I was still a normal or what most people consider to.
00:01:56 Mark Stouse
Be a typical.
00:01:58 Mark Stouse
Marketer right. And I sort of, I don't know, interested in performance, how how well marketing was performing in any given area, but it was a for me like it is for a lot of marketers, it was a defensive thing, right? I just I was using it to deflect criticism.
00:02:18 Mark Stouse
And to defend my budget. And that wasn't really something that Mark heard would accept from anyone, not just me, and not just marketers.
00:02:30 Mark Stouse
He was a very customer focused, very operations, focused, extremely smart guy who really, you know, was not opposed to holding your feet to the fire. And so marketing came under a tremendous amount of critique in terms of our inability to demonstrate.
00:02:52 Mark Stouse
On anything that anyone cared about and it wasn't that it wasn't effective, it was just that we had absolutely not a clue in the world how to express whether or not that was true and to what extent it was true and certainly had no idea at all how to pivot what we were doing in the face of.
00:03:14 Mark Stouse
Changing externals.
00:03:17 Mark Stouse
So I actually I got I got to a point of where I was sort of like saying to myself, look, I've either got to, like, do something to begin to fix this or I just need to go do something completely different. And I'm not famously or infamously, depending on the situation. I'm not really a quitter.
00:03:36 Mark Stouse
Right. And so I started, I kind of hauled out well, not kind of, I did haul out my college math textbooks, which when I graduated, I was absolutely thrilled to say goodbye to those.
00:03:51 Mark Stouse
Right. So there was like literally nothing in my background to this point that would have ever suggested that I would have ever gotten bitten by the math analytics science bug. I started formulating some ideas, you know, kind of got reintroduced to basic regression.
00:04:11 Mark Stouse
You know all that kind of stuff. And I and I went and talked to her, and I and then it all kind of started going right and.
00:04:18 Mark Stouse
Success sort of begets success. And because I was the middleman between a group of data scientists and the board and the executive team at HP, and this continued on this way for other jobs, I had to.
00:04:38 Mark Stouse
Figure out what my role was and my role was to ask them. What do you most want to know and what do you most care about and how do you want to understand it and what came back to me was and I'm going to express this.
00:04:57 Mark Stouse
In analytics terms, not in because they didn't really know how to express it for a data scientist, which is also my.
00:05:04 Mark Stouse
Role I'm the I'm.
00:05:06 Mark Stouse
The whisperer between the two groups, right? So he said. You know, essentially we don't care at all about 95% conference scores. Couldn't care less.
00:05:17 Mark Stouse
And in fact, most of the data in business is human performance, human behaviour, data and so that right there just undercuts the idea of 95%.
00:05:28 Mark Stouse
And totally like you, you kind of be lucky to get to 50 in many cases, right? What we care about is. And actually, I did this at at Honeywell, like an executive will say I make decisions every day worth millions and 10s of millions and hundreds of millions of dollars.
00:05:48 Mark Stouse
That if you were to model my decision after the fact might come in 10 to 20% confidence score and yet it's a pretty damn good decision most of the.
00:06:00 Mark Stouse
So really my what I want is I want you to get me into the 30s or the 40s because and I and because I a lot of these decisions they have to make on a repeating basis. So what they're actually trying to do is they're trying to make every time.
00:06:18 Mark Stouse
They remake the decision.
00:06:20 Mark Stouse
And they want it to be marginally better than it was last time. So to kind of exaggerate for effect, let's just say that you had to make the same decision every morning for 365 days, and your whole goal was that every morning was going to be 1%.
00:06:40 Mark Stouse
Or even a half a percent better than the morning before. So at 1/2 percent.
00:06:45 Mark Stouse
Per day compounded, you are now looking at close to 2000% annualised improvement. That's what they're after, that's.
00:06:54 Mark Stouse
What they care.
00:06:54 Mark Stouse
About and. So what really inhibits that is too much latency between the recalculations.
00:07:05 Mark Stouse
Like we went through this at at Honeywell.
00:07:08 Mark Stouse
Well, I mean, I was spending many millions of dollars on marketing analytics at Honeywell. We had to over hire data science talent in order to be able to get the latency on the recalculations down to the point where they were essentially happening at the clock.
00:07:27 Mark Stouse
Speed of the business.
00:07:29 Dr Genevieve Hayes
So you had multiple people on the same task, basically cycling.
00:07:32 Dr Genevieve Hayes
Through it.
00:07:33 Mark Stouse
Correct, because otherwise it's irrelevant. Otherwise you're always behind your weight. You know, you could be anywhere from 2 weeks to three months behind the actual decision. And so it's not even relevant, except if you're keeping score about the past, but nobody really cares about that. They want to be able to know what the future.
00:07:56 Mark Stouse
Might be and then to track your projection your forecast against actuals and be able to operate causal analytics a lot like a GPS.
00:08:10 Dr Genevieve Hayes
So it's basically make a small decision, check how you've gone correct and then keep repeating.
00:08:18 Mark Stouse
In in in relationship to a forecast.
00:08:21 Mark Stouse
So if you think about the.
00:08:23 Mark Stouse
Think about the GPS analogy. That's really what this is, right? That is. So GPS on your phone says, well, this is where you are. So you typically know that historical data on any problem, it will say, OK, this is where you say you want to go. So that's your.
00:08:39 Mark Stouse
That's your business outcome. In this case, it will give you 3 options on routes.
00:08:47 Mark Stouse
Which are forecasts, right? It's taking all the current data about traffic and weather and all kinds of stuff, right? And it's saying, hey, these are three viable ways to get to your destination, and each one is going to take you approximately this.
00:09:02 Mark Stouse
Amount of time.
00:09:03 Mark Stouse
You then pick one and you are cruising along.
00:09:07 Mark Stouse
Right. And it's tracking you and it's keeping track of all of these externalities that are either going to speed you up or slow you down.
00:09:16 Mark Stouse
Town. Or if you were, if if we're now kind of moving more into something like an aeroplane or a ship can push you off course. Right. The current push you off course. So being able to then keep track of all that and have a system that says, hey, we see a delta.
00:09:36 Mark Stouse
Opening up between the original forecast.
00:09:41 Mark Stouse
And actuals and we predict that this is going to continue to widen.
00:09:46 Mark Stouse
If you do.
00:09:48 Mark Stouse
And so now you can play.
00:09:51 Mark Stouse
With it, this is the way it works in proof you can you can war game responses to that, effectively exploring rerooting yourself with an explainable variance on time lag, or you know whatever and.
00:10:08 Mark Stouse
It allows you to stay reasonably optimised and so that is an extremely practical.
00:10:18 Mark Stouse
Illustration of the difference between data science and business and data science in a academic or research setting.
00:10:30 Dr Genevieve Hayes
So in a research setting, you're just doing it once and that's it, whereas in business you've got to keep doing those calculations and corrections.
00:10:38 Mark Stouse
I wouldn't necessarily say that you're just doing it once and and in Academy, but but the latency, the recall of the model.
00:10:47 Mark Stouse
Not unusual for it to be.
00:10:49 Mark Stouse
Six every six months every 12 months.
00:10:52 Mark Stouse
It can be.
00:10:53 Mark Stouse
Pretty extended and so that is if you're trying to run a business that where decisions are happening on, let's say, a weekly cadence that does you absolutely no good, you might as well just scrap.
00:11:08 Mark Stouse
Your data.
00:11:10 Mark Stouse
Because it's just not going to give you anything that ultimately will move the business forward. Everything will continue to be in the review mirror for the most part. A lot of our customers are actually highly experienced and advanced in what's called marketing mix modelling.
00:11:31 Mark Stouse
Which is essentially econometric analysis, is what we're talking about right here on Mark.
00:11:36 Mark Stouse
Thing and, but they do, they've done it the old fashioned way. They've done it, you know. And so these are big, big mega models that are recomputed every six to 12 months, sometimes every two years. And So what happens is is that, let's say it's every six months, the model is calculated and then it begins to age out.
00:11:57 Mark Stouse
And then six, six months later, it's recalculated.
00:12:01 Mark Stouse
But then it takes all the data science team doing the work anywhere from another 60 to 90 days to prepare the insights for consumption by people who have no data science background at all. And so it's now at nine months so.
00:12:21 Mark Stouse
They get the they get the results and even the forecasts are most of the forecasts are now in the past.
00:12:31 Mark Stouse
So operationally, it's just kind of a bust. And So what I what I learned is is that, you know, life is a series of numerator, denominator relationships and it's always that way and that is a scalable idea comes philosophy at a certain point and.
00:12:51 Mark Stouse
In this situation, the business the needs of the business are always the denominator. Business is not going to move to data science.
00:13:01 Mark Stouse
To accommodate data science, data science is going to have to accommodate business, and it doesn't mean that mathematical principles and mathematical laws get thrown out the window. That's not what that means. But if your business team needs an analytic every hour on the hour.
00:13:22 Mark Stouse
And you can only do it every week on the week you got a problem. You're gonna have to meet them where they are.
00:13:30 Mark Stouse
Another example of this is we figured out when we were developing a lot of the user screens for proof that if all we did was take the data science readouts on these models and render them in a beautiful way, they still would be large.
00:13:53 Mark Stouse
Unactioned able by most teams, most teams. It's really weird. There's a lot of research on this, right people who aren't data math type people, they have no problems with charts, but they have a psychological block with graphs. OK, so if you.
00:14:13 Mark Stouse
If you just give it, if you give the give them what essentially you would give a data scientist, they'll throw up their hands and go. What the hell am I supposed to do with this?
00:14:23 Mark Stouse
So OK, so if that's the case and your whole goal is to make a difference to make it better, help them make better decisions, then you're going to have to do something different, right? You're going to have to meet them where they are because you're not going to get, you're not going to convince them all or force them all.
00:14:43 Mark Stouse
To go through two semesters of data science classes at their local university, this is not.
00:14:49 Mark Stouse
So you just got to kind of think a little bit differently and most data scientists weren't trained to think that way, right? Another great example of this. So one of the things we're seeing a lot here in the United States right now, which is sort of reminiscent of what happened to enterprise IT teams.
00:15:09 Mark Stouse
About 20 years ago, after Y2K.
00:15:12 Mark Stouse
Is that data science teams are being moved under finance and a lot of companies and the reason for that is that when everything became unspooled about a year, year and a half ago, the C suites turned around to the data science to their CEO's right and said, OK, I know you've been spending millions of dollars and.
00:15:35 Mark Stouse
Lots and lots of people hours over the last five to seven years. Now it's time to earn your key. The problem was, is that almost all that time was spent on data management systems.
00:15:48 Mark Stouse
And they hadn't created the final step.
00:15:51 Mark Stouse
Like the analytics deliveries?
00:15:53 Dr Genevieve Hayes
It's actually getting something that the C-Suite can take action from.
00:15:58 Mark Stouse
Right, you can actually use right? And so it was sort of a a lot.
00:16:02 Mark Stouse
Of what they.
00:16:02 Mark Stouse
Had done was still kind of a road to nowhere and that really ****** *** a lot of sea suites, so these data science teams are being placed under the CFO not because finance believes.
00:16:18 Mark Stouse
That it can.
00:16:19 Mark Stouse
Teach data scientists anything about data science.
00:16:22 Mark Stouse
But to replicate what they did with it 20 years ago, very successfully and.
00:16:27 Mark Stouse
That is, change the.
00:16:29 Mark Stouse
Culture drive a far greater T shaped kind of approach into data science where so data science right now is an eye shaped approach, right? Because it's it's function. First it's math 1st it's analytics first and so they want to create a a situation where.
00:16:49 Mark Stouse
The CEO is a business leader who happens to have a great deal of expertise in data science and but who sees the data science through the business lens, not the other way around.
00:17:02 Dr Genevieve Hayes
So it's a means to an end rather than an end.
00:17:06 Mark Stouse
Absolutely. And which is, I think, a very business principle, right? I mean that's that's the way a business leader looks at almost anything. Even employee happiness is a means to an end.
00:17:18 Dr Genevieve Hayes
Yes, that's keeping staff turnover down basically.
00:17:23 Mark Stouse
Right. So I mean, you know and you can kind of you can, you know people have different people have different ideas about that and you know you can get very philosophical about it. But but from a very practical standpoint.
00:17:36 Mark Stouse
That's a unassailable truth, and it's sort of like gravity. You cannot like it all you want to, and it's not going to change for you.
00:17:46 Dr Genevieve Hayes
While you've been talking, I've been trying to figure out how all of this works technically, so these models you're describing are.
00:17:56 Dr Genevieve Hayes
The regression type models or are they time series type models?
00:18:00 Mark Stouse
They are multivariable, linear and nonlinear regression models that obviously use almost exclusively time series data, because that's what businesses collect. Parenthetically, this is sort of the great unanswered problem with AI.
00:18:20 Mark Stouse
Is that AI in businesses? Anyway? AI is a big data set of solutions. If a business doesn't have a lot of big data.
00:18:30 Mark Stouse
It's sort of.
00:18:32 Mark Stouse
Hard to implement a lot of AI, right? And so I mean we there are obvious exceptions, particularly regulated industries, aerospace, automotive, pharma, healthcare, those just a few, they're going to have a lot more big data, but still.
00:18:51 Mark Stouse
Even in those situations.
00:18:53 Mark Stouse
It's overwhelmingly technical or research oriented data. It's not big data about the business of the business. This is where actually what I I am one of the most profound ironies of the of this whole thing, right is that it's the old tried and true.
00:19:14 Mark Stouse
Multivariable regret?
00:19:16 Mark Stouse
Question that actually still answers about 80% of the world's questions and it the fact that it's not sexy, right? And not cool doesn't.
00:19:27 Mark Stouse
Change that fact.
00:19:28 Dr Genevieve Hayes
So you'd have all these observations of various variables at regular intervals of time, so I don't know. Hourly, daily, whatever it is.
00:19:39 Dr Genevieve Hayes
And they would predict something revenue or whatever it is. And from that you can work out. If I pull this lever then that will cause revenue to go up.
00:19:50 Dr Genevieve Hayes
And what you've got with proof analytics is a tool that basically is constantly refitting that multiple linear regression.
00:19:59 Dr Genevieve Hayes
Model in order to make sure that you don't end up with model deterioration, is that right?
00:20:06 Mark Stouse
That is correct. It's also re optimising.
00:20:10 Mark Stouse
Whatever it is that you're doing, a great way to think about this is that when when a client implements proof and begins to run it in the very beginning.
00:20:22 Mark Stouse
What they discover is the 8020 rule is alive and well, right? So anywhere from say, 20 to 35% of marketing investment or go to market invest.
00:20:32 Mark Stouse
Investment is either suboptimal or non performing, so it's waste and it's not hard to stop spending money that money, so they they they get an immediate benefit and then they can put that money, they can reinvest it in areas that where the model.
00:20:53 Mark Stouse
Says that you can get more if you spend more, you'll get more good stuff up to a point. You know the point.
00:20:59 Mark Stouse
Of diminishing returns.
00:21:01 Mark Stouse
And so then on an ongoing basis, net of relevant externalities, it will keep your wastage in the three to.
00:21:11 Mark Stouse
5% range.
00:21:12 Dr Genevieve Hayes
So you might discover that advertising on Facebook to one particular demographic is a waste of money. But advertising on LinkedIn.
00:21:21 Dr Genevieve Hayes
To a different demographic is resulting in an increase in sales, is that right?
00:21:26 Mark Stouse
Yeah, that.
00:21:27 Mark Stouse
Yeah, sure. I mean, another great example would be.
00:21:31 Mark Stouse
How is all of your go to market investments? So let's just say that that's marketing and sales together. How is that impacting average deal velocity, meaning how quickly you close deals on average, this is really important because there's a one to one.
00:21:51 Mark Stouse
Relationship between average deal velocity and cash flow from revenue.
00:21:55 Mark Stouse
So if you're closing faster, that means you're invoicing faster and you're collecting faster. And CFO's love that. If you've got 14 billion in revenue and you get it where it's moving 5% faster into the company, the CFO will be your best friend.
00:22:15 Mark Stouse
Wherever that's a great example of being able to figure out what the right mix is. So marketing is a non-linear multiple.
00:22:28 Mark Stouse
There of areas of linear business performance, one of which is sales.
00:22:35 Mark Stouse
In this case.
00:22:36 Mark Stouse
In plain language, it means marketing brings a level of leverage to sales performance that sales cannot self generate.
00:22:46 Mark Stouse
So the value of marketing is really the extent to which it is making sales more effective and more efficient. The efficiency of the effectiveness.
00:23:03 Mark Stouse
It's not just efficiency, because you can be very efficiently doing the wrong thing. So in practical terms, the way this a lot of times goes down is that marketing starts to really make sales very effective.
00:23:23 Mark Stouse
Very efficient to the point where they don't need as many sales guys as they once did.
00:23:29 Mark Stouse
Because there's sort of a built in wastage in that as well. So not only are they they're total close rate improving constantly but the speed at which they're closing is improving. So they're able to stack more deals per quarter into a given sales reps.
00:23:50 Mark Stouse
Per view, so their total cost of sales goes down, while at the same time driving much better outcomes. That is, that's an example of that.
00:24:01 Dr Genevieve Hayes
What sort of lag do you experience between when an executive makes a decision and when you can start to see the results of that decision in the?
00:24:09 Mark Stouse
Ah, see, that is a great question. So time lag is, in my view, the great enemy from the standpoint that if you don't know the time lag, you will never find the value. So this goes back to the fact that we live in a four dimensional world.
00:24:31 Mark Stouse
Three of which are physical, one is time, right?
00:24:34 Mark Stouse
So if you and I agree, we're going to go have lunch somewhere and you say, hey, I'll meet you on the 20th floor of a building at the corner of walk and don't walk. But you never tell me a date and a time you and I will never.
00:24:48 Mark Stouse
Meet it's sort of the same thing here, right. And so one of the great things about.
00:24:57 Mark Stouse
As an exploratory piece of work is that it generates a time lag relationship, right that allows you to see on a.
00:25:08 Mark Stouse
One to one.
00:25:08 Mark Stouse
Basis. What that what the strength and speed of that relationship might be kind of looks like.
00:25:16 Mark Stouse
As a way of.
00:25:17 Mark Stouse
Of vetting what relationships might be valuable in a regression model? And so in our particular case, both historically and on a forward looking forecasting basis, it's going to give you a stack rank that's perpetually updated at whatever this the cadence is that you.
00:25:37 Mark Stouse
We need it to happen that says out of all these things, these this is the most effective, the least effective, and this is the fastest to the slowest.
00:25:49 Mark Stouse
So it allows you to begin to really understand again on a multivariable basis because there's not one all up time lag going to show you how things are kind of. I mean it allows you to really think about marketing and sales as a investment portfolio that you're managing.
00:26:09 Mark Stouse
So the the specific answer to your quest.
00:26:12 Mark Stouse
Question is it really depends. It depends on the business. It depends on the headwinds and tailwinds that are happening. There are some things in marketing, for example like brand investment where the time lags are consistently pretty long, but.
00:26:32 Mark Stouse
The half life is also pretty long, so they.
00:26:36 Mark Stouse
They take a while to.
00:26:37 Mark Stouse
Build and then they hang around.
00:26:39 Mark Stouse
For pretty good length of time, right, there's a lot of residual value there, whereas in what's called demand, so demand marketing is like I'm sure like you like everybody else gets a tonne of email that you did not necessarily ask for, that's trying to get your business right. That's one example to.
00:27:00 Mark Stouse
The vote the average lag on demand, high quality demand programmes is going to be a lot shorter than it is on brand.
00:27:09 Mark Stouse
It's going to be stuff in the middle, so.
00:27:10 Mark Stouse
You've got to.
00:27:11 Mark Stouse
Kind of be able to really understand and manage what you're doing across a series of renewal cycles would be one way to kind of simplistically think about this at what cadence do I always need to renew?
00:27:28 Mark Stouse
The pressure in the.
00:27:29 Mark Stouse
System because if I don't renew the pressure in the system, all of a sudden on a time lagged basis that means that all of a sudden something is going to Peter out down here and it'll take a while to get it restarted. And I don't want to have to restart the engine. I just want to. But I also don't want to over stimulate the engine.
00:27:50 Mark Stouse
Either I don't want to spend more than I need to spend in order to get the result that I'm after. Does that answer your question?
00:27:56 Dr Genevieve Hayes
Yeah. Yeah, it does. So with regard to the Lags you could have in your model, you could have a variable that's some factor one week ago. And what will that have on revenue, but in some?
00:28:08 Dr Genevieve Hayes
Mother's in a different model. It might be one month ago or one year ago.
00:28:13 Mark Stouse
Yeah. It's it's highly related. Again, there's this is a mathematical principle almost so not no great surprise here, 60 to 65, maybe even 70%. I think if you talk to most data scientists, it would be sort of in that range, right of any given model situation is going to be externalities.
00:28:36 Mark Stouse
So this is highly related to what those externalities are and how strong they are. Either way, head, wind or tail.
00:28:48 Mark Stouse
Or cross one it works that way too.
00:28:51 Dr Genevieve Hayes
So a while back you were talking about the challenges of communicating the outcomes of this analysis to C-Suite executives, and you're saying how you couldn't just provide them with your standard Python output from fitting these models. So I take it we're talking about the.
00:29:08 Dr Genevieve Hayes
The fitted coefficients of the regression T values all those.
00:29:12 Mark Stouse
Yeah, using any use any of those words, OK.
00:29:18 Mark Stouse
Over it's over.
00:29:20 Dr Genevieve Hayes
If all those things are out of bounds, how do you communicate it to the C-Suite?
00:29:24 Mark Stouse
You talk about the same principles using business terms, so you talk, you know, like if you use time, I mean, I do occasionally use the word time lag or the phrase time lag.
00:29:35 Mark Stouse
Because most can grasp that, OK. But I will say how many people are like, particularly like early conversations, first conversations, right, I'll say, how many of you agree that great marketing takes time to be successful? I have yet to find anyone.
00:29:56 Mark Stouse
Even the most hard bitten CFO, who won't.
00:30:00 Mark Stouse
Agree with that stat.
00:30:02 Mark Stouse
However, that's not really the issue.
00:30:05 Mark Stouse
The issue is how much time, because if I have to the in the in the mind of a business leader, if I have to wait a really long time to see the payback, then on a time adjusted basis that the payback isn't as good.
00:30:23 Mark Stouse
And it's also exposed to a lot more risk.
00:30:28 Mark Stouse
Mainly from extra now.
00:30:30 Mark Stouse
So they look at longer term risk with a much more jaundiced eye than stuff that where the the feedback loop is really fast and yet they also know that if I mean I'll, I'll use this analogy because everybody gets it right. If you're running a paper.
00:30:50 Mark Stouse
Company and you don't replant the forest, you're gonna not be a paper company very long. There is a long lag component to every business.
00:31:01 Mark Stouse
We have to recharge the system.
00:31:03 Mark Stouse
So you got to be able to figure.
00:31:05 Mark Stouse
That out, it's one of the reasons why a lot of.
00:31:06 Mark Stouse
Distilleries they may.
00:31:09 Mark Stouse
Be that you.
00:31:09 Mark Stouse
Know their pride and joy products might be fine aged bourbon whiskey.
00:31:16 Mark Stouse
But the time lag on that is 7 to 10 years minimum.
00:31:21 Mark Stouse
They don't have a cash crop like vodka or gin.
00:31:26 Mark Stouse
They won't have the cash flow to stay open long enough to reap the benefits of the 1012 year old whisky.
00:31:33 Mark Stouse
So it's about balancing this whole thing. And so that's what they're after. That's what they want to understand. They honestly, in my experience and I think the experience of a lot of other people, they don't care about understanding the math like honestly, most of them are mathematical philistines.
00:31:54 Mark Stouse
They not only don't understand it, they don't want to understand it. They just want to understand that you know it and that you are standing behind it and.
00:32:04 Mark Stouse
That it's going.
00:32:05 Mark Stouse
To be good, it's going to be dependable.
00:32:08 Mark Stouse
That's really what it is. One of the things like the that we do at proof is we're always every time we do a product update, we retest the entire system using usually large amounts of healthcare data with the known outcome.
00:32:24 Mark Stouse
Because we want to make sure that everything stays aligned and so the goal is prove it better. Replicate the known outcome pretty much perfectly out to I think our standard is 5 decimal places.
00:32:39 Mark Stouse
Because, I mean, it's the same math, the math is the math. There's no proprietary math on this, right? So so you're kind of you should, you should have exactly the same outcome as somebody who did it on Stata. That is not the magic of proof. The magic of proof is being able to do it a lot.
00:32:59 Mark Stouse
Faster, a lot less expensively, a lot more scalable.
00:33:04 Mark Stouse
Really like a lot of the a lot of times these companies are paying several $1,000,000 a year for two models and like they're doing business in like 50 countries and yet they can't afford to support 50 markets with the same analytic. Well, that's crazy. So with proof.
00:33:24 Mark Stouse
They can totally do that. It's a it's radically afford.
00:33:28 Dr Genevieve Hayes
So going back to what you were saying a few minutes ago with communicating to these executives what I'm hearing from that is that rather than talking about technical stuff like coefficients and values, you might express the way you're communicating in terms of if you do this, it will have a.
00:33:47 Dr Genevieve Hayes
Tax dollar impact on revenue and the.
00:33:51 Dr Genevieve Hayes
Payback period if that would be Y years or whatever.
00:33:55 Mark Stouse
Right. And you're it's reducing your risk substantially because you're recalculating the model so frequently that if things start to change either in the quality of your execution or more likely externalities, you.
00:34:12 Mark Stouse
Pick up that chain.
00:34:13 Mark Stouse
Change very quickly and be able to respond as opposed to six months later, discovering that you are way off course and having to change. Actually, I grew up in a sailing family. We did a lot of long distance racing.
00:34:33 Mark Stouse
And I from a young age, was taught to be the navigator.
00:34:37 Mark Stouse
And I just happened to be old enough where when I was doing all that the my role was going through a whole bunch of technological upgrade. So first you know, because I learned where the sex and the chronometer and all that kind of stuff, right. And and then you kind of get sat NAV.
00:34:57 Mark Stouse
So it used to be when I first started out that you would, you would run a Plumb line on the chart, which was your perfect, hyper efficient course. And then every time you would reestablish your location, you are basically zigzagging back and forth, very eccentrically across this Plumb line.
00:35:18 Mark Stouse
One way to kind of think about that is the interior area of each one of those triangles that that make up the zigzag.
00:35:27 Mark Stouse
It's waste. It's wasted time wasted resources, bears against your ability to win the race. All that kind of stuff. Then all of a sudden with sat NAV, right, we had a perpetual feed on location and then magic happened because it integrated with autopilot and so.
00:35:48 Mark Stouse
The autopilot was taking the location information and constantly tweaking the position of the rudder relative to everything else, and so.
00:35:59 Mark Stouse
Instead of these giant zigzags back and forth across the Plumb line, we.
00:36:06 Mark Stouse
Called it the.
00:36:06 Mark Stouse
Snake all of a sudden it was these really shallow little, very, very mildly eccentric back and forth curves across the Plumb line. Lot less waste. We were one of.
00:36:19 Mark Stouse
The first boats.
00:36:20 Mark Stouse
To actually implement this and so for one whole season, we just kicked.
00:36:25 Mark Stouse
And then everybody else got the same technology and it was sort of mutually levelling like the average time that it took to complete one of these races dropped by about 15 to 20%.
00:36:39 Dr Genevieve Hayes
Oh wow.
00:36:40 Mark Stouse
Right. And so for like everybody, net of all other variables, everyone experienced that kind of efficiency gain. So it it meant that you could do more races if you chose to. It meant that you didn't have to have as much food. Food is actually really heavy. It slows the book down.
00:37:00 Mark Stouse
There were a lot of ways that people started playing around with the variables that were essentially improved by what I just described right 15 to 20%. So how are we going to use that 15 to 20% to actually get more out of it, right? Because if we get.
00:37:20 Mark Stouse
Like another five points faster man that's going to be awesome. We we're probably gonna win more. At least for a time. It's that constant optimization.
00:37:31 Mark Stouse
And it's not just about Esquires, it's it's actually very, very practical when you think of it through a GPS lens, as you're seeing the relationships right there, like more or less in real time, depending on how you define real time and certainly real time in the context of a busy.
00:37:53 Mark Stouse
And so you're able to adjust and constantly drive out.
00:37:58 Mark Stouse
The inefficiencies in.
00:37:59 Mark Stouse
This case, a lot of the inefficiencies are you're eliminating those eccentric zigzags.
00:38:07 Dr Genevieve Hayes
A few minutes ago, you mentioned that when you do the updates of this, you test it on health data and a lot of what we talked about is about marketing data. What other contexts could you apply a tool such as this?
00:38:22 Mark Stouse
Actually I mean so we have to, you know, a cardinal rule of a start up or a scale up businesses that you got.
00:38:29 Mark Stouse
Focus. So we focus on go to market because it is so.
00:38:34 Mark Stouse
It's such a huge.
00:38:35 Mark Stouse
Issue there's a.
00:38:36 Mark Stouse
Lot of money to be made there.
00:38:38 Mark Stouse
But the reality of it is, is that proof is agnostic on use.
00:38:44 Mark Stouse
So we do have customers that use.
00:38:46 Mark Stouse
It for other things.
00:38:48 Mark Stouse
And if you stop and think about it mathematically, it has to be agnostic.
00:38:53 Mark Stouse
Because you're bringing all of this dissimilar theoretically unrelated data together, consumer confidence data is comes from a completely different mindset, a completely different source and marketing data. The fact that one impacts the other is is a.
00:39:14 Mark Stouse
Analytical equation.
00:39:15 Mark Stouse
In reality, but otherwise these are very disparate.
00:39:19 Mark Stouse
And so to accommodate that proof, just like the underlying regression has to be extremely flexible. I mean, climate change is a great example of this. I mean, there are so many different variables to consider that you and and most of them you.
00:39:38 Mark Stouse
Just you'd be.
00:39:39 Mark Stouse
Outside of a an analytical construct, a model you'd be hard pressed to say that one has in any way related.
00:39:50 Mark Stouse
The other thing that's really I think a great example that climate change shows you is that the importance of time lag.
00:39:58 Dr Genevieve Hayes
Yeah. And you did that fantastic video that illustrates that.
00:40:02 Mark Stouse
Yeah, I mean.
00:40:02 Mark Stouse
It's it's a I think that one of the reasons one of the reasons why so many people have resisted the science around climate change is that they walk outside until just recently last several years it it got a little it's gotten a little hairy, right. But for years and decades.
00:40:23 Mark Stouse
They would walk outside and they'd say.
00:40:25 Mark Stouse
I don't see any change.
00:40:27 Mark Stouse
And what? Oh, scientists got to be out of their mind, right? I'm not saying it's not getting hotter here. It's not getting a lot colder here, right? I mean, maybe we'll have a cold, cold winter, but we always have had cold winters or we've always had hot summers. That kind of stuck.
00:40:43 Mark Stouse
And then all of a sudden, because and again this is the non linearity at play all of a sudden it's incremental, it's incremental, it's incremental and then boom and hockey sticks. And that's what you're seeing.
00:40:57 Mark Stouse
And that is also something that you see in go to market. You see in business, right is it will feel while everything is in flight, that's the time lagged bit, feel like nothing is happening and business leaders will get very freaked out by that because.
00:41:18 Mark Stouse
Their mindset is we launched this marketing campaign in Q1 and I should be able to see an effect of that in Q1 or early Q2 and in some businesses that's.
00:41:31 Mark Stouse
True and in other businesses, it's not even.
00:41:33 Mark Stouse
Close to true.
00:41:35 Dr Genevieve Hayes
I always see it as being like the activation energy in chemistry. You know how you'd have the reaction and it would have to get to some sort of chemical point before. Yeah, it would suddenly do some cool thing.
00:41:43 Mark Stouse
Critical mass.
00:41:48 Mark Stouse
Yeah. No, I think that's.
00:41:49 Mark Stouse
A great example as well. I think that human beings are fundamentally uncomfortable with the extreme multi variability and nonlinearity of life. So we try to create constructs that simplify that.
00:42:07 Mark Stouse
In some cases it appears and unfortunately for longer, for long enough period of time where it seems to give it credibility, it appears that it's right and then all of a sudden it's not right. It's way out.
00:42:22 Dr Genevieve Hayes
An exponential relationship looks like a linear relationship. Initially it's only once. It has that Inflexion point.
00:42:26 Mark Stouse
Yes, correct.
00:42:29 Mark Stouse
That's exactly it. Right. So I mean it's just a yeah, absolutely. I mean it, it's a I think that that one of the ways that analytics can make you a better person is that it shows you the truth. Small. I'll even go with small T truth.
00:42:49 Mark Stouse
It will show you the facts about what is actually happening.
00:42:54 Mark Stouse
In a given situation, and there are some takeaways that are inescapable. For example, how little we actually control, it's actually kind of mind boggling. And yet it's still we are still really, really, really, really, really important. Great example would be.
00:43:14 Mark Stouse
You're surfing the waves.
00:43:17 Mark Stouse
Absolutely. The wave is easily 70 to 80% of that equation and you're not ever going to have any control over it no matter what you do. In fact, only the most nutty narcissist in the world would ever dream. They could control the.
00:43:31 Mark Stouse
But what's also important is the quality of your feedback loops that you build with between yourself and waves. Plural.
00:43:41 Mark Stouse
And your ability to actually surf, to shift, know how to shift your weight on the board, know how to manoeuvre the board in relationship to the wave. Again, numerator, denominator. You're not the denominator, but that makes all the difference between.
00:44:02 Mark Stouse
Finishing with a flourish on the beach and wiping out, so it's impossible to say the analytics would never say that you are absolutely immaterial and totally a victim. Nor would the analytics ever say that you are a master of the universe.
00:44:19 Mark Stouse
Both are actually totally narcissistic views. That's something actually that I was taught when I was in kindergarten. But I'm ashamed to say it took analytics to kind of sink the hook of that idea. I shouldn't have been such a tough nut.
00:44:39 Mark Stouse
But but I was.
00:44:40 Dr Genevieve Hayes
You see it from time to time in the world when someone who should be insignificant makes a massive change. You know, the mother of a child who died in a horrific accident, who takes action and draws awareness to some problem in the world. You know, things like that. These are people who, in the grand scheme of things.
00:45:01 Dr Genevieve Hayes
Shouldn't matter, and yet they can manage.
00:45:04 Dr Genevieve Hayes
To implement massive changes in people's behaviour and all sorts of things like that.
00:45:11 Mark Stouse
Absolutely. And and also you know the other thing to always remember is that it is a ripple effect. So what that mother does is magnified by other people in other situations that she does not control.
00:45:28 Mark Stouse
But who, who come alongside her, so to speak, either with intent or just situationally, and take her efforts and catapult them?
00:45:39 Mark Stouse
Forward and upward that is also a statistical reality. It's an analytical reality.
00:45:46 Dr Genevieve Hayes
Yeah, that's what my mom always said, which was whatever you're thinking. Odds are there are tonnes of other people who are thinking exactly the same thing and just no one's willing to say it.
00:45:57 Mark Stouse
That's right. I think the other thing too is that I don't want to lose this because I think it's really important. Data scientists are human, just like everybody else, and in their own lives, they like instant gratification. It's one of the reasons why they they all hate data prep because there is no instant gratification.
00:46:19 Mark Stouse
And yet they don't act many times, as though they understand that business leaders want instant gratification, too. And so this whole idea that they can walk in to a boardroom 3 months after a decision was made and say we have all these.
00:46:39 Mark Stouse
Fantastic insights that we have plucked about that decision and have everybody care.
00:46:47 Mark Stouse
Is just wrong. The world actually in so many ways, is driven by your ability to satisfy the desire for instant gratification. It's a it's not just an economic in the case of proof, right, not just an economic set of considerations or even a speed to market sort of construct.
00:47:10 Mark Stouse
It's the ability to know when I want to know what's going on and that is.
00:47:16 Mark Stouse
Very human, extremely human and flawed, and yet you're not going to ever successfully win that contest.
00:47:25 Dr Genevieve Hayes
Is there anything on your radar in the AI data and analytics space that you think is going to become important in the next three to five years?
00:47:33 Mark Stouse
Tur, I think that if we don't do it exactly the right way, we're going to, we will have a generation of people who become so reliant on it that they, they lose the ability to think critically.
00:47:47 Mark Stouse
And I think also in many use cases, it's going to involve a level of it's going to involve us giving up a level of privacy that all of us are going to have real concerns about. So one of the things that I have a personal interest in and I when I'm not doing what I'm.
00:48:09 Mark Stouse
What I'm doing right now I I read a thought and do a lot of primary research in the area of.
00:48:15 Mark Stouse
The history of innovation in the period of time leading up to the Renaissance comparing and contrasting northern Italy and southern Germany, both hot beds during that period of time of innovation. But who approached it very differently and it is. It's one of those things where.
00:48:35 Mark Stouse
If you ask ChatGPT to write a research paper, fully footnoted academic style research paper.
00:48:44 Mark Stouse
On some aspect of that subject matter, which I'm probably really well versed in, where there's a lot of content on the web and yet it's still fairly arcane in about 4 minutes it will spit out this absolutely incredible looking paper. And if you knew very little.
00:49:05 Mark Stouse
About the subject.
00:49:06 Mark Stouse
You would say you read it and you go wow, this sounds great to me.
00:49:11 Mark Stouse
If you have a fair amount of knowledge of the subject, what you're going to find today is that it is riddled with inaccuracies and inconsistency.
00:49:21 Mark Stouse
And total fabrications. So I actually did this exact what I'm describing, I did it and much to my amazement, so I saw some footnotes that I was completely unfamiliar with. And I'm like, wow, how exciting. I want to actually go find these and read them.
00:49:41 Mark Stouse
Right.
00:49:41 Mark Stouse
They weren't real. They didn't exist, right and.
00:49:41 Dr Genevieve Hayes
Didn't exist.
00:49:46 Mark Stouse
Then the problem is with a lot of this.
00:49:48 Mark Stouse
Tenant that's produced by Gen AI is sooner or later, it's going to be re crunching, if you will, data that's was generated by AI to begin with. And so the problem gets compounded and if we don't really take so.
00:50:08 Mark Stouse
It sounds like that.
00:50:09 Mark Stouse
I'm really anti AI, I'm not but I am aware that this is never a technology, is never a bind.
00:50:16 Mark Stouse
Mary topic. It's amoral. What you do with it and how well you bear against the risks versus the rewards of that technology is everything. Nuclear is a great example. Can it cost effectively heat and cool 10s of millions of homes around the world?
00:50:37 Mark Stouse
If we chose to sure, can it also, you know, extinguish life on this planet? Yes, it it can. I think that. Have you seen the movie Oppenheimer?
00:50:49 Dr Genevieve Hayes
No, I haven't.
00:50:50 Mark Stouse
It's a. It's a. It's a. It's a phenomenal film as history it's it's probably one of the best historical films I've ever watched. I thought one of the things that really captured was all the these scientists at Los Alamos knew that there were great risks associated with this. They didn't exactly know.
00:51:11 Mark Stouse
What those risks might really be?
00:51:14 Mark Stouse
But they knew that there were real risks, but they had nothing to compare it to. And so when they did the first detonation, their first reaction was.
00:51:25 Mark Stouse
I wish that we could go back in time and univent this. Now that's a hell of a statement. Famously, you know, in his diary a couple days later, Oppenheimer wrote that, you know, he floated that famous Hindu text about I've become death, the destroyer of worlds. Right. Right about that.
00:51:44 Mark Stouse
Time that all that that was coming out in the movie and all this kind of stuff, you also had the foremost father of AI, the guy at Google.
00:51:54 Mark Stouse
Finally, resign from Google and go.
00:51:56 Mark Stouse
Whoa, whoa, whoa, whoa, whoa, whoa, whoa.
00:51:59 Dr Genevieve Hayes
This is Jeffrey Hinton.
00:52:00 Mark Stouse
Right. So you're so you're kind of like going. Hey, you know, that's probably somebody we should listen to. This is not necessarily a. Again, it's not a binary thing. It doesn't mean that we just jump.
00:52:15 Mark Stouse
AI and somehow say, well, we're we didn't really do that and we're just going to act like we're going to wall it off over here because that's not real. But now that we have done what we've done, we need to have some really, really, really good rules around how it's going to operate just like.
00:52:33 Mark Stouse
We have with nuclear.
00:52:34 Dr Genevieve Hayes
What final advice would you give to data scientists looking to create business value from data?
00:52:40 Mark Stouse
You may not think of yourself in this way, OK, but you were taught to be a member of a cult of precision.
00:52:49 Mark Stouse
That is fundamentally incompatible with most businesses. It does not answer the questions that they care about at the speed or latency that they need it. And when you walk in with results that are so heavily caveated because that's what you would do with fellow data scientists, business leaders look at you and go.
00:53:10 Mark Stouse
Well, if you're that unsure, then why are we even talking? So if I had to give any data scientist.
00:53:20 Mark Stouse
A2 Word piece of advice I would say be relevant before anything else. It's not not telling you that you need to absolve yourself of mathematical principles or anything like that. That's not it.
00:53:35 Mark Stouse
But you're not being asked.
00:53:37 Mark Stouse
To tack it down to 95% or better.
00:53:40 Mark Stouse
That's not the goal here. If you think it's the goal, you will never and I underscore never be successful in business analytics. You might be successful in a research portion of that business, but in terms of analysing the business of the busy.
00:53:59 Mark Stouse
Thus, you will be a bust if you can't do it fast, you won't be successful if you can't be understood by people not like you, you won't be successful. I I find myself in this very, very interesting place. So I sit in three circles.
00:54:19 Mark Stouse
Professionally, I sit in the CEO, CFO Circle. I sit in a marketer circle and I sit in a CDA CDO circle and I can hang out and hold my own with all three groups. But I play the alter ego to each.
00:54:37 Mark Stouse
Like with CDO's, I mean I love them and I and and data scientists in general, I mean the whole tendency to debate how many angels can dance on the head of a pin. I totally love that kind of conversation. So I can.
00:54:51 Mark Stouse
Sit with them.
00:54:53 Mark Stouse
And that since I'm not a normal marketer, not a normal business leader, right. I.
00:54:57 Mark Stouse
I can geek out with all of them, and yes, I have learned a lot about.
00:55:04 Mark Stouse
What they do?
00:55:04 Mark Stouse
But I am not a data scientist myself.
00:55:09 Mark Stouse
But I think that I can probably represent what they do to other audiences better than they can themselves. Most of the time. I will then move over to the business leaders and the marketers and I will represent the point of view of data scientists to them.
00:55:29 Mark Stouse
To challenge them to say, hey, you know what, neither one of you guys know any math other than how to add, subtract, multiply and.
00:55:41 Mark Stouse
So you have when you ask for certain things, you have no idea what you're asking for you, you know. So it's part of it is, I don't really deal with the technicalities of data scientists with their with these other audiences. And I I or vice versa, I am.
00:56:00 Mark Stouse
I'm managing people's expectations. I'm helping them all understand each other better, and then we have a software platform that kind of instantiates that those principles, right?
00:56:14 Mark Stouse
OK. To the extent that I do conversations like the one that you and I are having or I stand on the stage or whatever, it's because of that perspective, I it's not because I'm a, you know, a whiz bang mathematician. I mean I I do love it, but I'm not. That's not my that's not my thing.
00:56:34 Mark Stouse
Not what I'm asked at.
00:56:36 Dr Genevieve Hayes
And as I said to you before the interview, the thing that made me most want to have you on this programme was.
00:56:43 Dr Genevieve Hayes
That video you did in the middle of Sweden, where you're drawing the analogy between climate change and what proof analytics does? Because I thought that was the best way I'd ever seen someone explain an analytical tool.
00:56:57 Mark Stouse
Well, thank you. I mean I, I.
00:56:59 Mark Stouse
Will say this I think.
00:57:00 Mark Stouse
That the best.
00:57:02 Mark Stouse
I mean, almost every analogy will break. The only ones that won't break are the ones that feel like analogies, but actually aren't. They are representations of the same thing, and so I think that that's where.
00:57:17 Mark Stouse
Climate change, things like that, or as analogies that people can relate to.
00:57:22 Mark Stouse
Totally do not break because it it's undergirded by the same principles. The GPS is an analogy does not break because progression actually is part of the way GPS actually works. One of the things that I used to do a long time ago was I didn't care about whether the.
00:57:43 Mark Stouse
Analogies were at some point I was just trying to connect with the audience and get them to see it to good perspective. As soon as I got into data science, and particularly as CEO proof, I only work. I only use analogies that are not really analogies, they just look.
00:58:02 Dr Genevieve Hayes
So for listeners who want to learn more about you or get in contact, what can they do?
00:58:07 Mark Stouse
So I think so I'm I'm a I'm very active, maybe even hyperactive on LinkedIn. That is a super easy way to reach me. My presence on LinkedIn is a non sales function I am.
00:58:22 Mark Stouse
Here to talk to people, to help people. I don't track them down. I don't chase them down. I don't bug them until they buy, you know, a licence to prove. I don't do any.
00:58:31 Mark Stouse
Of that stuff.
00:58:32 Mark Stouse
If you want to have that kind of conversation, I can totally make that happen.
00:58:36 Mark Stouse
For you. But that's not the conversation you'll have with me.
00:58:40 Mark Stouse
Say the only reason why I say that is you don't need to worry that if I if you reach out to me that I'm that you're going to wish that you'd never done so that I would say LinkedIn either a DM and LinkedIn or or just comment under one of my.
00:58:54 Mark Stouse
Comments and just.
00:58:55 Mark Stouse
Say hey, can we connect, you know?
00:58:59 Mark Stouse
I would say.
00:59:00 Mark Stouse
Certainly email, although it's getting less and less, but my email is Mark MARK dot status S as in Sam T as in Tom Ouse at proof analytics. Just like it sounds dot AI.
00:59:20 Mark Stouse
And then you.
00:59:20 Mark Stouse
Know our website is proof analytics dot AI, so those are three really super excellent ways.
00:59:26 Dr Genevieve Hayes
And I'll put the links in the show notes. So thanks for joining me today. Mark.
00:59:32 Mark Stouse
Oh, I thoroughly enjoyed.
00:59:33 Mark Stouse
It you know, I mean it's.
00:59:36 Mark Stouse
It's I think the most exciting things, by the way. As a historian, I would say, this is also a historical truth. The the most exciting things are in the in the way that other things converge together right to create something different. That is, that's where throughout history.
00:59:56 Mark Stouse
You see the biggest for good and not so good. I I love the the the fact that I sit right now at this intersection.
01:00:05 Mark Stouse
Is a joy to me, right? And I I hope that that has come.
01:00:08 Mark Stouse
Through in this.
01:00:09 Mark Stouse
Conversation I I just thoroughly enjoy the living heck out of this.
01:00:13 Dr Genevieve Hayes
Oh, I totally enjoyed this conversation, so I'm really grateful that you made the time to be here.
01:00:19 Mark Stouse
Thank you so much.
01:00:20 Dr Genevieve Hayes
And for those in the audience, thank you for listening. I'm doctor Genevieve Hayes, and this has been value driven data science brought to you by Genevieve Hayes Consulting.