[00:00:00] Rajeev: You cannot make strategic decision from attribution. You cannot make operational decisions from mm. MM, right? And also, you can't test everything though. We are all for experimentation culture for most of the brand. It's an overhead, right? [00:00:12] So you should be testing what absolutely needs testing. So this, this multi-model approach. Primarily enables actioning for the marketers, where everything else is abstracted away, but making sure that everything is transparent as well. So, uh, e every assumption that has gone into your model building has to be transparent, should be open for scrutiny. [00:00:32] ​ [00:00:59] In This Episode --- [00:00:59] Phil: What's up [00:01:00] everyone? Today we have the pleasure of sitting down with Rajiv Naer, co-founder and chief product Officer at Life Site. Life Site is a unified marketing measurement platform that allows marketers to leverage advanced measurement techniques. And this episode we explore digital astrology in the attribution illusion. [00:01:17] How quasi experiments can help marketing teams, how to make sense of conflicting measurement methods. When will AI agents run marketing measurements on their own? All that and a bunch more stuff after a super quick word from one of our awesome partners. [00:01:32] ​ [00:02:30] Phil: Rajiv, thanks so much for joining the show today. Really appreciate your time. [00:02:34] Rajeev: Uh, thank you Phil. Uh, super excited to join you on this podcast. Looking forward to it. [00:02:39] Phil: We've chatted with a lot of, uh, attribution measurements, uh, focused folks who are experts in this space, building in this space, or consulting, freelancing in this space. And I, I was really interested in a lot of the work that life Lifestyle is doing around causal machine learning and your unified measurement framework. [00:02:58] Um, so I'm excited to, to take a deep [00:03:00] dive in the year with you, but maybe we can start with, uh, your thoughts on, on, on attribution. It's the polarized topic with a lot of marketing ops folks. Today. When I chat with listeners, it continues to be one of the top topics that people are confused about or trying to figure out. [00:03:15] The Dilution of The Term Attribution --- [00:03:15] Phil: So you've actually said that much of marketing attribution as we know it is scientifically flawed. In a LinkedIn post, you wrote that marketing has evolved a lot from the TV Mad Men Days. To TikTok, but our measurement methods are kind of frozen in time and we're chasing vanity metrics instead of true effectiveness. [00:03:34] Maybe shout about that a little bit. Like can you unpack why you think traditional attribution is broken? Do you still think that multi-touch attribution or click-based attribution is still useful for something today, like content effectiveness or path to purchase? What are your thoughts there? [00:03:49] Rajeev: Uh, sure, sure. Phil. Uh, so the, the way attribution is, uh, used in practice today in the industry is to essentially, uh, do user tracking, right? So attribution [00:04:00] has come to mean user tracking right now. The problem with this is when we approach any measurement solution, we are primarily interested in two things. [00:04:08] Are we able to understand the cause and effect? Uh, so if, if you see some, if you observe certain events, are we able to understand the causes? That is leading to these events. So that's the, uh, in the truest sense, that's the meaning of attribution, right? So if you look up the word attribution dictionary, it has a cause and effect connotation associated with it. [00:04:27] So that is missing in the way attribution is practiced today. So that's problem number one. Problem number two is, uh, the, the marketing ecosystem today is pretty complex, right? So you have got a bunch of different factors. You've got your paid media variables, your own media channels, uh, your, um, uh, giving out a lot of running, a lot of promotions, giving a lot of discounts, making a lot of price variations and all of that, right? [00:04:52] So it's a, a, a a pretty complex. A process. So for you to make any meaningful optimization decision [00:05:00] across all those factors, you need to understand what is each of these factors incrementally offering you? Right? How are they incrementally driving your outcome? So the attribution, the way it is practiced today, doesn't solve for causality, doesn't solve for incrementality, right? [00:05:15] So that's our primary problem with attribution, right? And, and that's why we believe that it's, it's broken. Uh, the meaning of the word in itself is, uh, uh, is sort of changed in today's marketing context to just mean tracking. Um, and it, it has got diluted significantly. [00:05:34] Phil: Yeah. Yeah. I, I feel like what you're saying is part of the problem is in the word itself, like the definition of the word. Like, I've worked with plenty of companies where if we throw around the word attribution, it's synonymous with multi-touch attribution. We've had some guests on the show that kind of explained that as like, attribution is a bucket of figuring out what is driving effectiveness, what is driving revenue for marketing. [00:05:58] And MTA and [00:06:00] click-based stuff is just one of the methods of doing that. And last touch, first touch, multi-touch, uh, incrementality, MMM, causal stuff. Like all those things are different methodologies under the bucket of attribution, but we throw around the word attribution. Everyone instantly just like goes right to MTA and MTA is gonna tell us what drove revenue. [00:06:19] And like you said, a lot of folks have, have also pointed out the issue with cause and effect there. And, and even like correlation and some of the signals we're getting, like it's, it's behavioral analytics. From what we can see, and that's like the part where I find it's really hard to defend With MTA, what we can see is diminishing all the time and what we can stitch together in terms of like a user's online journey is also diminishing. [00:06:45] So yeah, MTA is really hard to defend, but. Like you built Lifesight seven years ago, right? Like you've been in this space for a long time. Uh, the initial product that you guys were working on was third party measurement to solve user journey [00:07:00] attribution and to solve for some of that like journey stitching. [00:07:03] You were using ID graphs, uh, basically you were in the credit distribution game, right? Like some of the stuff that, that you're talking about. And at some point chatting with customers or, you know, researching more stuff about this space, you discovered that this didn't lead to enabling businesses to make better decisions. [00:07:19] We weren't like signaling what was driving revenue. There was no causality and what you were doing. So maybe chat about like that pivot, like what led you to discovering that baselines and incrementality were missing from the approach and, and you kinda went towards that area. [00:07:34] Rajeev: Sure, sure, sure. So, [00:07:34] To Measure What Caused a Sale, You Need Experiments --- [00:07:34] Rajeev: so let me first unpack this attribution problem a little bit more. Okay. So, um, I think you, uh, very clearly sort of articulated the technical limitations in, uh, in, in having this user journey built. Uh, so there are a bunch of privacy concerns. Uh, you can't, uh, track all the, um, user exposure, all the user touch points. [00:07:56] Uh, now, um, most of the retail [00:08:00] businesses are omnichannel, right? So you've got. Uh, your digital media, social media, uh, offline spend. There are a bunch of other offline events. Your sales channels are also, uh, some of it is in your direct to consumer website. Uh, then you have product placed in marketplaces. [00:08:16] You have, you have your offline stores and things like that. So it's very hard for you to technically build the user journey. But let's assume that somehow you manage to do this, right? That you have the most perfect user journey available. You have got all your touch points of all your customers properly captured. [00:08:32] Um, now, now the question is, uh, what, what do you observe in this particular data? Right? So we have come through that journey, right? We have been, as you rightly said, when we started up, uh, we wanted to solve this attribution problem. And our philosophy was that we will try to build the most comprehensive user journey possible across both first party and third party data, building the most exhaustive identity graph and all of that, right? [00:08:57] So, e even once you have [00:09:00] all the data that you need. What you would finally see in the data is a lot of spurious correlation, Right. [00:09:06] So let me give you a couple of examples. So your bottom of the funnel would always be highly correlated with your outcomes, Right. [00:09:13] Because in the whole journey, more People are [00:09:16] engaging with your bottom of the bottom of the funnel right. [00:09:18] before they transact. [00:09:20] So you will find a lot of very high correlation between certain parts of this data with the outcome that you are interested in optimizing for. Um, now from this correlation, we need to glean out causality, right? So we, we have all, all all heard this cliche that, um, a correlation does not imply causality. [00:09:39] But the other side of the same equation is there is no causality without correlation. So correlation is a necessary condition, but it's not the sufficient condition. So, so causality is essentially correlation plus bias. Can we somehow manage the bias so that we could interpret the observed correlation as causality? [00:09:59] [00:10:00] It's a very hard problem to solve only from, uh, multitask attribution data. Let me give you a couple of examples here. Okay. So, uh, let's assume that there's this one physical store that you have and you have kept a billboard right in front of the store. So every user that walks into your store, they see this billboard. [00:10:20] Now can we say that this billboard is driving the walk-ins? Now, once these people are in your store, say that you are offering some discounts, and a lot of people are redeeming these discounts and they're making a purchase. So they were, uh, showing high intent by walking into your store. Is this coupons, uh, or, or discounts incrementally driving more outcomes. [00:10:42] So these are very difficult problems to only address from your journey data. Perhaps if you have two stores, you could keep a billboard in front of one store and not in front of the other. And that would give you a meaningful lift number. Uh, if, if you have, again, two stores, you could give discounts in one and [00:11:00] not in the other. [00:11:00] And, and here you are making some interventions and measuring, uh, measuring a difference. But if you look at attribution, it just shows a lot of correlation. It's very hard to info causality from, from this data, right? So that's the fundamental problem with, with attribution. Now, uh, we have had similar issues when we started the attribution journey at, Lifesight. [00:11:25] So, uh, we were in fact helping a lot of brands measure the out phone, billboards and things like that. So some of these questions actually came to us. Do we know the incremental contribution of these things? So we started working on, uh, incrementality testing back in the day in 20 19, 20 20. Our approaches were different. [00:11:44] Uh, we, uh, used to create, uh, we used to have exposed, uh, devices and then we used to do a lookalike modeling and we used to create synthetic control devices, and we used to measure the lift between these two. So approaches were different, but as part of our, [00:12:00] uh, research into the space, we sort of, uh, moved to more identity agnostic way of measuring lift. [00:12:06] That's with geo test and things like that. Uh, right. So that's, that's been, that's been the journey. Yeah. [00:12:13] Phil: Very cool. Yeah, it, it, it's cool to hear, uh, why you went down that path. I think that a lot of folks in this space, uh, like some of your competitors haven't had that light bulb moment yet. Like they still sell MDA products and they still claim that we can stitch the whole journey together. And I feel like, you know, we can chat about this later too, but like a lot of the MTA analogies are from folks that are just used to MTA, just being the thing that you get to show the finance team. [00:12:45] And your senior management team that, you know what I did last month, the money I spent like this, it, it drove to, to this much revenue. But you mentioned geo tests, so I, I wanna unpack that a little bit. We've had some folks, uh, chat about this. Um, but so incrementality [00:13:00] tests, geo tests are kind of a part of that. [00:13:02] Um, [00:13:03] Why Most Teams Get Geo Testing Completely Wrong --- [00:13:03] Phil: a lot of the criticism for incrementality tests and, and, and geo tests that there's often a large cost associated with it. Um, it's not a $0 endeavor, especially like if you're a huge company and you've got a massive monthly ads budget, you wanna figure out if that, if those ads are incremental or not. [00:13:21] You, you have to turn 50% of them off. 50% of $20 million is $10 million per month. Like that, that's a huge cost there. Obviously smaller for, for, for startups. Um, but you know, the thing with geo tests is that. It's not a true incrementality tests, right? Like randomization is the core element of control groups and treatment groups, and in a geo test, we're handpicking the regions and not doing it like we would in, in, in a mass market. [00:13:48] Incrementality tests, maybe chat about some of the limitations of incrementality tests and, and geo tests. What are your sir? [00:13:55] Rajeev: Um, sure. So, so there's this thing called. Um, fundamental problem [00:14:00] with causal inference, right? Uh, so before we get into geotechs, let me try to explain that particular problem. Uh, so for example, in an organization, uh, so you have launched a new, uh, policy. Right? Now you want to measure if your employees are happy about this, right? [00:14:15] But there's only one reality. You've launched the policy. Right? Now, you are measuring this, right? You don't have the counterfactual of in another universe where you didn't launch this policy, right? So that counterfactual is never visible. In geo testing cases, we are creating a counterfactual. So we pick a few geographies and we are creating a counterfactual, which is a set of geographies, a set of another set of geographies, and we are making an intervention in your treatment cluster. [00:14:48] And the other set is what is used as counterfactual. So you are assuming that. Um, uh, from the past, these two set of geographies were, uh, very similar in the way they were contributing your [00:15:00] revenue or whatever outcome. Uh, so you're assuming your counterfactual or your control group will continue access because you're not intervening there, right? [00:15:09] But you're making some interventions in your treatment cluster. Now you could, you could measure this difference. Now, most of the pushback that we, uh, get for geo testing is, uh, the pri the primary concern is that it's, uh, costly, right? 'cause most of these tests has to run for a period of three to five weeks, right? [00:15:28] So it's a slow process and it's a very rigorous process. It, it's a lot of overhead on the marketeer. 'cause they've got a lot of other things to worry about. Now they need to spend three to five hours running a test. And it has to be, you have to babysit the test, right? You have to control for a lot of things. [00:15:45] So there's opportunity, cost. Uh, now the other factor that you mentioned, you will have to. Scale up or hold out your investment on certain channels, in certain geographies, and these geographies might be contributing a significant chunk of your revenue. But for those things, [00:16:00] there are statistical solutions. [00:16:01] So we could create a test where your opportunity cost is the lowest possible, right? You could, we could pick the group of cities or states which perhaps only contribute to five to 10% of your revenue. And we could, uh, we, we could run a test in the shortest possible time, but making sure that we don't compromise on the statistical power of the test. [00:16:22] So there are ways in which some of these things could be addressed, but having said that, it needs more commitment from the marketing team right, to, to, to run these tests and to get these reads back to your, uh, optimization system. But having said that, uh, this is a better way to measure as well. Here, here you have got more control. [00:16:44] Uh, this is slower process, but you could adopt the learning. Uh, with more confidence, uh, right. Unlike other, uh, measurement system. Uh, so the framework that we have in mind is, so you've got a fast system of measurement. The fastest is [00:17:00] being attribution perhaps, but it is less accurate and less actionable. [00:17:04] Then you have, uh, models, right, that you could create on your historical data, uh, which is something that you could create in a few days to a couple of weeks. Uh, now the problem with the models is you won't have high confidence on all the reads that you get it. So for example, if you are having 10 input variables and you're trying to understand the impact of all of this. [00:17:27] Some of these input variables could be spars. You could have certain data challenges around multiline and things like that, which means you cannot interpret the reads on those variables with high degree of confidence. So that's when you should move to testing, right? So you, you, you, you, you should, uh, first have a process in place where you could generate intelligent hypothesis so that you are using up your bandwidth the right way, right running the right test, the most important ones. And, for [00:18:00] everything else, you could have COE experiments, which is where your model sort of comes in. [00:18:06] Phil: And, and like the, [00:18:07] Quasi Experiments Help You Test Without Tanking Your Quarter --- [00:18:07] Phil: the quasi experiment stuff is where you're talking about like minimizing the opportunity cost of, of some of these, uh, these tests. 'cause like the, the part that I've struggled with doing this in, in practice is that like, it sounds really good to minimize the opportunity cost marketing team is kinda reassured a little bit, but when you dig into the minimal detectable effect of, you know, how long you need to run this to get the right sample size, then like the, the, the opportunity cost is still kind of growing and, and you're looking at like, oh, how, how close are we to get like that, that sample size to me? [00:18:41] Maybe try to bet that a little bit, like the minimal detectable effect variable in, in this whole like incrementality testing. How do you think about that? [00:18:49] Rajeev: true, true. Yeah. So, so there are, there are a couple of, um, couple of approaches, right? So primarily what we are trying to do is we are trying to infer causality, right? We are trying to do causal inference. [00:18:59] Phil: Mm-hmm.[00:19:00] [00:19:00] Rajeev: Uh, and there are broadly two ways in which we could do this. One way is, uh, what we call quasi experiment, or it's also popularly known as natural experiment, which means your historical data in itself would have enough variations in it for you to infer causality from it. [00:19:18] Right. So the first, uh, entry point to incrementality test or incrementality reads is to create models on your historical data and see if you can infer causality out of it. Uh, so this is where your MMM system comes in. Now, coming to your question on the incrementality test itself, uh, yes, it's very important to design the test in such a way that you get conclusive outcomes out of this, right? [00:19:48] Uh, so let's take a, let's take an example. So say there is a particular brand, which is spending a million dollars a month and the brand gets, uh, $5 million, uh, out in [00:20:00] revenue, right? So this is five x, your blended ROAS is five x. Now, this particular brand wants to test a new channel, right? So the next month they introduce a new channel, they spend just $50,000. [00:20:12] So now you spend this 1 million and $50,000 and they see a revenue of 6 million. Okay, so this is a test. I've just added a new variable, but this $50,000 is giving me 1 million of delta, which is a 20 x row. So the question is, was this a good test? Was 50, was that additional 1 million the result of some other seasonal factors? [00:20:35] Right? So this is an example of longitudinal study or time testing, right? So that 1 million is perhaps because it's December or it's uh, whatever, it's your peak period, it's window and you're selling window clothes. So this $50,000 is not driving that. So you should design a test so that, uh, the, the investment should primarily, um, make sense that it should be able to drive an effect [00:21:00] that is detectable. [00:21:01] And also you should factor in all these seasonal things. So that's where geo testing is important because geo testing is not this time-based testing. In geo testing, you are taking December and you're picking two set of geographies that work very similarly. On all the previous Decembers for which you have data, Right. [00:21:20] And then you are creating, adding a new channel for say, one of those geography cluster. Now, Yeah, you, could make a meaningful inference. Um, so, so, so testing, as I said, is, is perhaps, uh, the best approach, but it's a very rigorous approach. Design of testing is what you should get absolutely right, uh, because rest of the testing, which is to deploy the test, get the read back, uh, and, and compute the lift, all of those are trivial. [00:21:49] In some ways. You could easily operationalize that. But if you get the design wrong, uh, then you would end up having a test, which is not conclusive. You have wrong reads out of it. [00:22:00] And there are a bunch of different problems. Now, uh, the opportunity cost minimization that I spoke about, uh, uh, was that when you design the test, you could perhaps design the test, which is of shorter duration. [00:22:10] But having said that, you will have to invest more. If you want to reduce the duration. Uh, but if you, if you want to run a powerful test that is mostly a slightly longer duration test, lower spend, but it would take more time to do, but you could compliment or, or augment this n day testing process with the natural experiments, right? [00:22:31] That's where MMM sort of comes in, right? If you have 10 factors for which you need to measure incrementality, you need not test all 10, you, you, the first responsibility is to generate the right hypothesis that warrants testing And mm, MM is a very good hypothesis generator. Uh, yeah. Uh, [00:22:51] Phil: Yeah, you, you get a hypothesis from MM and then you go off and do an experiment to validate the insights or, or what the MMM is is telling [00:23:00] you. You, [00:23:00] Why Causal Inference Matters More Than Ever in Marketing Measurement --- [00:23:00] Phil: you mentioned causal inference there a a few times in, in our chat, and I'm curious if you can unpack that for, for listeners. It wasn't new term for me just a couple months ago. [00:23:10] Like causal machine learning takes causal inference a step further when we're talking about this in the incrementality context. Instead of like using statistical methods to identify causal effects, causal machine learning uses machine learning to estimate causal effects. Can you break that down for us? [00:23:28] Like, is it true that causal ML is a way to move beyond simply identifying correlations to understanding the underlying mechanisms that drive outcomes? Basically allowing us for more robust explainable models. Break that down for us. [00:23:42] Rajeev: Yeah. [00:23:43] Um, Yeah. [00:23:43] a a, absolutely, absolutely. So, um, a Sal Inference is an umbrella LA and, uh, the most popular way of doing cosal inference that perhaps everyone has heard about is the clinical trials when it comes to testing medicines. Right? Uh, so let's take an [00:24:00] example. So there is, uh, a medicine that you want to test whether it cures, cures someone of a disease. [00:24:05] So you've created a treatment group of people. Uh, you give them the medicine, you wait for a wait for some time. And then there's another group of people, uh, to whom you give a placebo. Uh, you wait for say a few weeks and see if the people who got the medicine are, are they better off than the other group? [00:24:23] And, and let's say you get good results. Right? Now, if you look a little deeper, perhaps your treatment group was all female age between 25 to 45, and your control group was all male, perhaps age less than 25, right? Which means they were inco comparable to start with, right? So this the case of can we info causality from correlation? [00:24:46] You're observing co correlation, but in this case you cannot because there's bias associated with it. How to de-bias. You should have both cohorts with similar conversation, right? Uh, uh, that's where randomization comes in, right? So if, [00:25:00] if you're testing this to understand the efficacy of this drug in Canada, then your treatment and the control group should. [00:25:07] Have, uh, uh, the right representation of Canadian population, right? That's when it is, uh, that's when it's compatible. Now, this is causal inference. This is causal inference where we are controlling for the bias by treating or by designing an experiment, uh, properly and running the experiment for a few weeks and making some inference out of it. [00:25:28] Now, the truth about marketing is, uh, everything in marketing is Cosco in some way because we don't have a laboratory setting where we could test Facebook's efficacy, right? It is getting tested in the real world, right? That's problem number one. Problem number two is once a drug drug is tested, you are, you have discovered a universal truth about that drug, but that's not true about Facebook's performance for your business, right? [00:25:56] That is going to change next quarter, the quarter after that. So. [00:26:00] The testing here, you're not discovering any universal truth. You are, uh, discovering that Facebook is working for you today in this season. You have to repeat this in another quarter, right? So from that perspective, everything in marketing is slightly quasi cosell always. [00:26:16] Now, uh, in Cosell ml, what we are doing is we are, I mean this is a very imp important approach, and geo testing essentially does this, instead of taking individual cohorts, we take geography cohorts and, and that has got its own utility, right? Because geo testing is a common framework, which can do apple to apple comparison between Facebook and Google. [00:26:36] Google and TikTok. 'cause it's a common testing framework. You, you cannot get something similar out of platform live studies and Facebook, right? Because it's sort of a black box. Those, there are a lot of documentation. It's still a bit of a black box. So you can't compare Facebook to Google by just relying on Facebook and Google platform, live study. [00:26:52] So that way geo testing is, uh, is very, very, very useful. Now, coming to causal ml. Uh, so there's this, [00:27:00] uh, field of causal ai, right? Uh, the whole philosophy here is we could, uh, find, uh, smart experiments hidden within your historical data, right? So, uh, and that's a faster way to observe a few things. And a lot of cases, this is the only ethical way of doing things as well. [00:27:19] For example, um, if you need to understand the impact of smoking in cancer, right? We can't create two group of people and get them, one of them to smoke. Uh, right? So that's not ethical. So we'll have to infer things from the data that's out there. And, and this is, uh, a Nobel Prize winning idea. Uh, in, in, in 2021, the Nobel Prize went to this causal inference on observational data, Right. [00:27:44] So that's when this term called natural experiments sort of became Very very popular. Now, the problem with most of the algorithms that is emerging out of the cost AI spaces. Uh, they don't have the kind of maturity, I would say when, [00:28:00] uh, compared to the ML algorithms that's very popular in the industry. [00:28:03] So, for example, if, if you're familiar with supervised learning or regression based approaches, right? So there you have certain framework to understand accuracy, understand fit in terms terms of R square, uh, you could do, um, back testing, right? You, you will get the ground truth data, create a model and test the model on some holdout data. [00:28:22] So there's some framework that is very evolved and matured in the ML side, which is not that evolved in the SAL AI space. Uh, so what we are doing is we are adopting certain approaches from the Sal AI space. Uh, one specific approach is this thing called structural causal modeling, which is we, when we see all the data, we try to first, uh, impose certain assumptions on how these variables are interacting with one another. [00:28:48] And on top of that, we apply regular ml, right? Could be basin in regression, regression, whatever, but. What this enables us to do is, when it comes to MMM, [00:29:00] there are a lot of, uh, implicit assumptions in data transformation in the way the variables interact with each each other and all of that. So all those assumptions can be properly codified in, into the causal graph, and that becomes a very useful, uh, uh, communication layer between the modeler or between the data scientist and the marketer, right? [00:29:23] So in the causal deck, we could, uh, we could have things like, uh, so, so your top of the funnel variables, they are driving your Google brand search spend, right? So Google's brand search spend is not an independent variable. You, you tend to spend more in Google brand search if your TV ad is working really well for you, because a lot of people would come and search for your brand. [00:29:44] So there's this interaction, right? If you think of this from a regression perspective, these are just two variables, and the assumption is that they're independent of one another in any regression modeling, right? There's this. Assumption called, um, inde independently and identically distributed assumption, right? [00:29:58] So we assume that they [00:30:00] are, um, independent of funnel, which is not true, right? They're dependent, your, your click based variables. Uh, if you're paying, uh, per click, uh, you would end up more click if you are top of the funnel is really working for you, right? Um, so these assumptions are first encoded in some SAL language, and then we use regression approaches to understand the true impact. [00:30:22] So this way, uh, some of the time tested ML approaches in mm MM, right? Which is your, um, uh, regression based approach essentially, right? The different ways in which you could transform the data, you could do time sales decomposition, and a bunch of different things are brought into the world of causality. [00:30:37] And you, you have a hybrid solution. Eventually, perhaps a better causal AI algorithm might evolve, right? In which case you perhaps don't need causal ml. You could do everything in causal ai, but currently that's not the case, right? Um, so, so I mean, that's. Uh, I, I, I, I hope it all made sense. So that's how, that's the whole spectrum of causal inference. [00:30:58] You have experiments [00:31:00] and you have natural experiments, right? But natural experiments has got its own limitations as well. So if your data is sparse, so you have a new variable, [00:31:09] Phil: Right. [00:31:09] Rajeev: uh, which is very sparse, the model would struggle to truly understand its impact, that's when you should go to the experiment, right? [00:31:17] ​ [00:33:18] Phil: Very cool. Like the thing that stands out to me, and I I appreciate your, your, your, your detailed breakdown there, like the, the difference between. How causal AI thinks of experimentations is instead of going out and doing natural experiments in market and figuring out, you know, the design of that experiment and, and, and babysitting it, like you said in your previous answer, we're instead looking at historical data and trying to uncover hidden experiments that are within data that we already have, and then we can infer causality based on that. [00:33:54] Is, is that, is that the right way to think about it? [00:33:57] Rajeev: absolutely. Absolutely. Yes. [00:33:59] Phil: [00:34:00] Very cool. And so like we, we talk a lot about like these different methods, right? So like marketing operations folks, like are just hearing about causal ai maybe for the first time. They're just like scratching the surface on like, what is MMM, what is simplified? [00:34:15] MMM models. They're trying to like turn the leaf on, on MTA and trying to go more towards incrementality. Like there's so many different methods and this idea of like triangulate triangulation, like figuring out. You know, there's, there's limitations with all of these methods, right? And knowing them is super important. [00:34:34] And then like a lot of folks recommend that we still do all of them. And then we try to figure out what are the best signals from all of these things. And you guys will like, spend a lot of time thinking about this. [00:34:45] Why You Need a Unified Measurement Stack to Make Better Decisions --- [00:34:45] Phil: You're calling this like the unified measurement framework where you're advocating for like multimodal or like unified measurement approach. [00:34:53] What does that mean for you exactly? Like what does that look in practice and, and what do we do with that information if a bunch of [00:35:00] these different methods are all pointing at different solutions? Like, MMM is telling me that this thing is really good and then Causable AI is telling me that that thing is actually really bad, but this thing is really good. [00:35:10] Like, what do we do with all that, that information? [00:35:13] Rajeev: Yeah. [00:35:14] Um, so, so, uh, so when it comes to marketing optimization, right, we are, we are trying to make, uh, decisions at different levels, right? So certain decisions are very strategic, so you are thinking a few quarters down the line. Certain decisions are, uh, very tactical. You're worried about this month and the next, and certain decisions are very operational, right? [00:35:36] You, you are worried about, uh, today, tomorrow, this week, next week, right? So for each level of this ing you need different capabilities. So that's why all these different techniques are very important, right? So, so you, you can't use MMM to decide whether you should scale up or scale down. A creative today, right? [00:35:57] Uh, and, and that level of operational [00:36:00] ing is becoming more important by the time, because today, marketers can create many different creative variations very easily, right? So if your incrementality testing says Facebook prospecting can be scaled up, then the uh, uh, uh, question is, okay, I've got this 2030 creatives, how do I manage my budget within these creatives? [00:36:18] So for some of these, you would need multitouch attribution, right? So, um, uh, uh, though we have spoken against multitouch attribution, this is past one use case. So we are not saying that multitouch attributions like absolutely of no utility. Uh, but there are some of these use cases whereby you, you could make your operational decisions based on touch-based attribution, but the scope is very, uh, very, very limited. [00:36:44] So when we talk about unified marketing measurements approach, uh, uh, we are sort of bringing all these techniques and we are enabling seamless orchestration between. Uh, between each of them. The most important thing, um, that gets, gets [00:37:00] missed out in this discussion is there is a layer of abstraction on top of all of this. [00:37:05] So, when a marketer visits a unified marketing measurement system, uh, though we talk about how robust, uh, our modeling us, how we design experiments and all of that, finally, what a marketer see is a neat planning tool. Not even a dashboard, a planning tool to which you could ask certain questions, right? [00:37:23] You could say that, uh, how, how could I drive my revenue up to X dollars next quarter? I've got Y dollars to spend. What is the right mix? Uh, what would happen if I add extra affiliate partners? What would happen if I change the price? So, uh, one major criticism is that, uh, this creates, um, uh, analysis paralysis, right? [00:37:47] You have got a lot of reports and all of that, but the point that a lot of people misses. Uh, these are specific details. Finally, what a marketer see is a planning tool right now, if the marketer is really interested in [00:38:00] knowing how the modeling works, how robust, uh, how do you, uh, uh, model for ad stock? How do you model for saturation and a bunch of different things? [00:38:07] How do you focus, how course we could get into the most specific, uh, statistical approaches that sort of powers everything. Uh, but unified measurement is primarily an action driven system, right? So analysis parais could happen in any framework. Uh, so there's this very, uh, very popular book that has sort of influenced me, uh, quite a bit. [00:38:28] So this book is called Decision Driven Analytics. It is written by a, a wha professor called Stefano Pan, right? So what it says is you start by thinking about the decision that you want to make and then use the data to, to, to see if it supports fit support. Within what conference range, right? So if you, if you think that you want to change your price approach analytics with that decision in mind, uh, so it's not that you see a lot of dashboards and then you're lost in it, [00:39:00] right? [00:39:00] Uh, so, so from that perspective, one, one thing, this is an absolute necessity, right? You cannot make strategic decision from attribution. You cannot make operational decisions from mm. MM, right? And also, you can't test everything though. We are all for experimentation culture for most of the brand. It's an overhead, right? [00:39:18] So you should be testing what absolutely needs testing. Um, uh, uh, right? So, so this, this multi-model approach. So we, uh, only very recently we started calling this unified marketing measurement. But even in early 2023, we had this, this multi modeling approach where we had attribution, we had incrementality test everything in one platform. [00:39:38] So this is a necessity, but it's very important that, uh, unified measurement vendors like us also. Uh, primarily enables actioning for the marketers, uh, right where everything else is abstracted away, but making sure that everything else, everything is transparent as well. So, uh, e every assumption that has gone into your model [00:40:00] building has to be transparent, should be open for scrutiny. [00:40:02] Your algorithms should be transparent. How do you focus? Should be transparent. Documentation should be there. So it shouldn't be the case that we just take the customer data, create a model, and just plug some insights in the ui, right? The process of creating a model should happen in the platform. So that's when UMM become transformational. [00:40:18] Uh, otherwise it's just a BA tool. Uh. [00:40:20] Phil: Right. No, it is super interesting the way that you're, you're, you're talking about this in a sense that a lot of folks think of measurement as this like retrospective tool to look at, like what worked and what drove revenue. You're trying to like spin that in. Measurement should be in this unified framework, a forward linking, uh, forward thinking tool on helping you figure out business decisions and what to do next. [00:40:48] Focusing on actions. What, like, how, [00:40:51] Stop Asking One Model to Answer Every Question --- [00:40:51] Phil: how do you handle customers that come to you and they're just like, we want to use Lifesight to figure out, like our business question is. Where do we invest more dollars in? We want to know what is working, what is driving revenue so that I can cut the shit that isn't working and I can double down on the stuff that is working. [00:41:10] That's my business decision. All of these different ways of, of measuring what is working, what is not working to inform that decision. You're saying that like there, there are drawbacks to all of these methods, but there are specific spots that are operational for, and, and better tuned for certain methods and, and things that are better tuned for. [00:41:29] Is that the right way to think about it? How, how do you think about that? [00:41:32] Rajeev: Yes, yes. Ab ab, absolutely. So I will, uh, perhaps take a very precise example, uh, right. Let's say, uh, there is this one particular brand, uh, who has, uh, their own direct to consumer website in Shopify and say they also sell products in Amazon, right? So you've got two sales channels. Now you've got a bunch of factors that is driving your direct to consumer revenue. [00:41:56] So this is the price of your product, all your paid media [00:42:00] activities. Your own media, right? Which is your email, your SMS and things like that. And perhaps you are earning a lot of discounts and promotion, uh, in, in your Shopify website, right? So you've got a bunch of factors. Now in Amazon, perhaps you have got your, uh, uh, Amazon ads that you would see when you go to the Amazon website, or perhaps you would also have the Amazon ad displayed across the internet in different other Amazon network publishers, which would redirect back to the Amazon. [00:42:28] So in these cases, what we would do is we will create a model, uh, explaining your D two C revenue. We'll create a model explaining your Amazon revenue. Uh, but in these cases, there are lot of interaction that could possibly happen between this, your, um, spend that is primarily, uh, meant to drive your D two C uh, traffic or treat D two C conversion could create some halo effect on Amazon as well, right? [00:42:53] People might also go and search the product. In Amazon after seeing a Facebook ad, vice versa is also [00:43:00] possible. So we create two models by, uh, applying absolute modeling rigor at each level. And once we get good models, we can easily merge this model and create a master model, which is for your revenue focused, right? [00:43:12] Overall revenue focused. So you could do focus and optimization for Amazon, you could do focus and optimization for D two C, or you could do focus and optimization for your overall revenue. So this is the starting point. This end day process is just a few days away for any, any brand in most of the cases. [00:43:28] Once this is done, the insight, besides the insight right, there will be a bunch of things that you could get out of this. Most important thing is that we should also raise a few warnings, right? If we see that your Facebook prospecting and Google top of the funnel are very highly correlated. Then model would always struggle to actually distinguish one from the other, right? [00:43:50] That's a modeling reality. So it's very important that we flag this to the user, right? Though we have got some reads we are not very confident about, say, Google top of the funnel because it is [00:44:00] highly correlated with, uh, Facebook, right? If there are affiliate variables, MMM model mostly struggles with affiliate variable because, um, there are many different affiliate business models, right? [00:44:11] So in One of the business model, you end up paying a commission to an affiliate partner after the sales has happened. So here the spend happens after the revenue, right? Which, which is opposite to the MMM assumption, which is you first have a investment or spend and then you get a revenue. So in affluent, so there are cases where we should raise a few warnings. [00:44:31] So these warnings are hypothesis, right? So we are essentially telling you that. For those variables. Before you make any addition, you should ideally be testing. But having said that, there are a lot of other variables which are already well understood. So you can start scaling up or scaling them down with a very high degree of certainty and confidence. [00:44:52] But some of the other variables needs testing and testing from the modeling perspective is an extremely important [00:45:00] part because what testing also provides you is a process called calibration, right? So calibration is whereby we have, uh, uh, understood a particular coefficient with a very high degree of confidence. [00:45:12] No, we can tell the model that when, when the model tried to find the coefficient by its own methods, you try to minimize the distance between model generated coefficient and the experiment generated coefficient for that period. For ways the test has happened, there's a process called calibration, right? [00:45:28] Um, uh, and, and that improves. So even if you could calibrate just one variable. It could calibrate the, it could improve the modeling accuracy across the spectrum for everything. So that's what makes experiment a more important thing. It in itself could enable a lot of ing, but once it comes back to the model, through the process of model calibration, that improves the end day measurement framework, right? [00:45:50] So this is one part of your triangle, right? Where mm, MM and experiment sort of talks to each other. Then you have attribution as well, right? Where, uh, the [00:46:00] biggest problem with attribution is this duplication problem, right? So when you get a revenue, uh, all your RAD platforms is taking credit for that. So finally, your attributed revenues 20% more than your actual revenue. [00:46:12] So when you bring attribution to this thing, uh, you need attribution for this, uh, operational listening, right? But at the same time, we could calibrate attribution also from certain multipliers that we could generate from the MMN experiment side. Uh, so the major controversy associated with this process is that it's not. [00:46:32] Scientific, right? That getting this multiplier from mm, MM and applying that to attribution number is not very scientific. Uh, I, uh, agree to that in some, some extent. Uh, but what we are essentially doing is we have gotten, so the reads that you get from MMM is an average radio. We are trying to understand the average impact of Facebook. [00:46:51] So that average number, we are percolating down to your more granular numbers like ad set, ad id, creative and all of that. So that way it has its [00:47:00] utility and it, it, it can do deduplication of your attribution numbers as well. So that's how the system works, right? Where each of these methods complement the other, and overall you have, uh, a, a, a, a better program, right? [00:47:14] A marketing program or marketing orchestration between all of these things. And most importantly, you have system to make strategic decisions, tactical decisions, and operational decisions, right? [00:47:23] Stop Using Attribution to Validate Your Job --- [00:47:23] Phil: One of the popular criticisms for having measurement vendors do a bunch of different methodologies and allow users and marketers to see different answers for some of the same questions is that the turnover for CMOs in tech is so short that you know, they to to, to survive in their jobs, they need to show their impact on revenue. [00:47:46] So they buy a tool to validate their existence versus actually informing what is working, what is not working. And so like, if, if they buy a unified, uh, measurement vendor approach, and it introduces this like, element of [00:48:00] variability, um, maybe you don't like what the MMM results are showing you, you just tweak this lever here and you give more way to MTA and here, boom. [00:48:09] Now your campaigns and the money that you've advocated for shows things a bit more favorably. How do you think about that versus like. Being true to what the data is telling you versus allowing your buyers to tweak the levers to justify their existence and and their investments. [00:48:28] Rajeev: Yeah. Uh, so there's this, uh, saying that you could, um, torture the data and it'll confess to anything. but with, uh, with, uh, with the unified marketing measurements, sys sort of systems, uh, if you torture the data, it'll show in the ui, right? So the only way in which you could create a model that, uh, agrees with. [00:48:52] Your assumptions or, or, or that sort of agrees with your story. Uh, all those configurations are there in the UI [00:49:00] open to scrutiny, right? So the assumptions that you have incorporated could be scrutinized by everyone. So it's, it's a very open, uh, system, uh, that way. And, and, uh, the way I think about this is, so far everything that we discussed it, it's uh, uh, we could classify that as the hard problem of attribution, right? [00:49:20] So we have been talking about statistics, the technicalities on how to create the model and things like that. But there's a soft problem of attribution, which is that every other business function is becoming more deterministic and more optimized. So your HR function, it function, product building, supply chain, everything is very deterministic. [00:49:42] It's becoming better year on year, but marketing is what is getting, becoming more, more challenging. And more probabilistic. So, uh, your business leaders, your CFOs, your CEOs are sitting on a bunch of very deterministic number. There's not nothing probabilistic about a loss of $3 [00:50:00] million, right? So they're sitting on top of those red numbers and then the marketing outcomes with a lot of uncertainty and probabilistic interpretation of what's happening. [00:50:11] So that's a soft problem, right? So the only way to address that is to be very transparent about the communication, very transparent about how you have arrived at those problems. And the reality of reality is that it's uncertain, right? Right. So, and there's no way in which we can wish that away. Uh, uh, and, and it, this is something we see a lot of marketers coming to terms with. [00:50:36] So your, um, uh, mm m numbers are, will never be as glamorous as your attribution numbers, especially. It'll never be as glamorous as your. Self-reported attribution numbers, right? Uh, because we are primarily interested in incrementality, but the important thing about these incrementality numbers is they're easily verifiable, right? [00:50:56] Uh, you could easily prove or disprove that these [00:51:00] numbers are wrong by adopting a few techniques that you could create a forecast you need not adopt the recommendation. You could. So that's what we recommend all our brands as well, right? When we create our first model, we, you can use the model to create a forecast. [00:51:14] You tell the model that this is what you have planned to invest for next month or next two months. The model will create a forecast, see if the forecast is accurate, right? If the model is able to forecast well on what you've already planned for. Right then the model has generalized well on your historical data. [00:51:30] Once you have build, have built enough conference on focus, then go to the optimization part, which is asking the reverse, right? Which is, I need to read somewhere. Gimme the mix. So this approach, uh, there's no one shot solution. It's not that the first model, you have absolute conference on the first model, but that we can have a program or a process where over a couple of months you can improve the model. [00:51:54] You can build a lot of conference on the process, right? That's possible. [00:51:59] Phil: Very [00:52:00] cool. I I like how you turned it, uh, the softer, like the, the people human problems with attribution and measurement. Like, it makes me think of in, in, in all the companies that I've worked at, like we never graduated or gravitated towards measurement minimization. We were always obsessed about measurements. [00:52:20] And [00:52:21] Stop Measuring Everything --- [00:52:21] Phil: I'm curious to get your take if you think that, you know, you're, you're selling a measurement solution, but do you think that people are too obsessed about measurement? Like, do we really need to focus on measuring every channel, every campaign, every creative, like. Every piece of content, every email nurture, like everyone is trying to justify their existence within the marketing team. [00:52:43] And it's important to, to show your impact on metrics and stuff like that. But like from a marketing operations, marketing leadership standpoint, like do we need to obsess about measuring every single thing? Like do you think teams should focus on significant channels and campaigns, [00:53:00] significant sources that like burn real money and can scale when you like, add a bunch of money to it, increase the budget and insignificant in the sense that like, you know, organic or email or direct traffic, like those things are meaningful, but like you can't really scale them by adding, you know, 500 k next month. [00:53:21] Like how much do you attribute to this idea of like, should we just measure the stuff that matters And like, how do you think about that? Like what matters in terms of measurement your men? [00:53:32] Rajeev: Yeah. Uh, so I'll, I have a slightly different perspective. Uh, perhaps what I would say is there are some major factors, and it's absolutely important that you understand their impact with as much accuracy as possible, right? Because say your 10 percentage of your spend goes to that one particular factor. [00:53:53] Uh, so it, it's very important that you're very confident that it's driving new results. So, uh, so you, [00:54:00] you should perhaps measure everything, but the rigor that you apply, uh, could vary, right? You need not apply the same rigor on some of the newer channels or at a very granular, creative level, right? You could, you could, uh, have a slightly more relaxed approach towards that. [00:54:18] Now, the truth is, uh. Uh, you, you can only control what you can measure, right? So if you want to manage something, control for something, then you should have some measurement process, um, for that. Uh, for brands which are just starting up, uh, uh, which does not have a lot of spend, you're still in one channel. [00:54:40] In those cases, perhaps you could have a very scrappy measurement program in place, and that should solve most of your questions. It has got its own pitfalls as well. The, the, uh, experiment example that I uh, mentioned, right? So if you introduce a new channel, uh, if the experiment is not properly designed, you'll make wrong inference.[00:55:00] [00:55:00] From that, uh, uh, but you could be slightly mindful of these things and you could perhaps manage everything yourself as well. But once things get complex, uh, right, once you have, um, more channels, some of which are offline, you have more sales channels, some of which are, again, not trackable. So when things get to that level, you'll perhaps need a more formal program of measure measuring things. [00:55:25] But to, to, uh, come back to your question, of course, the core function is to drive business value, right? You've got a lot of things to worry about. So you should perhaps, uh, uh, not put the same effort of measurement across everything that you're doing. You, you could prioritize what needs, uh, a lot of effort, a lot of focus, right? [00:55:49] And what could be done with, uh, the Quas approach and where mal attributions perhaps still, okay? Okay. To, to use. You could, you could perhaps [00:56:00] balance the bandwidth or balance the rigor accordingly. [00:56:04] Phil: Very cool. I appreciate your thoughts there. Uh, time is fine in, in our interview. Uh, this has been super fun. Uh, Rajiv, I got two last questions for you. Uh, it's, you know, on the show we talk a lot about AI and, and AI agents on top of, of measurement, and there's, there's a really cool overlap between those two areas. [00:56:22] Um, [00:56:22] When Will AI Agents Run Marketing Measurement on Their Own --- [00:56:22] Phil: there's a lot of buzz with like agentic AI and, and marketing and systems that can make decisions and act with minimal human or marketing input. Uh, we know that like Lifesight has been talking about marketing intelligence agents. Um, do you envision this like near future where an AI agent is autonomously running marketing measurements and, and optimizations? [00:56:45] Like you talked a lot about the unified framework where you're focusing on actions and, and business decisions. How automated is that gonna be in the next couple years? How are you thinking about that future? [00:56:56] Rajeev: Uh, yeah, so, so we are super excited about what, uh, the evolution [00:57:00] that, that we see around us in terms of Gen AI and agent d, AI and costal ai right. In all of this. Uh, and we recently launched our, uh, first version of marketing intelligence agents. We call her mia. Uh, and, um. And, and that's the, uh, uh, phase one of, uh, our, uh, adoption of, uh, HND ai, right? [00:57:23] So the way we see it, uh, internally in Lifesight is there are two AI verticals that we, uh, work on. One is the measurement ai, right? Everything that is needed to make the end day measurement process more robust, which is to adopt cost AI and some of the, um, more latest, um, uh, things that's coming up in that space. [00:57:44] Second part is NDK. So the primary purpose of, uh, uh, adopting LLM and agents on top of the measurement stackers, it would make things easier for the marketer to interact with the models, because for most of [00:58:00] them, uh, it's, it's a bit of a learning curve, right? So they now need to get used to some very complicated, complicated statistical concepts. [00:58:08] So it's not easy for most of them. So if we could create a conversational layer so you could converse with this data. Uh, and, and thereby you could understand a lot of things. And while we build these conversational layer, we are also making it, uh, uh, making sure that we become very transparent about the reasoning of these, uh, models or, or the, or the agents, right? [00:58:31] So when you ask some questions around, uh, uh, do we think we need creative refreshing Facebook campaign, A, B, C, right? Or tactic A, B, C, when the agent sort of replace, uh, with an SN no, there has to be proper reasoning, and that reasoning will need some human validation for some time. That's part number one. [00:58:53] Second is even when we generate, uh, next best action recommendations or optimization recommendation, [00:59:00] at this point, we don't go and automatically change it in your downstream platforms. It, it waits for. Uh, uh, a user approval, right? So in near term, I. [00:59:10] think most of these agentic system will have a human in loop, um, uh, uh, because the cost of hallucination and wrong reasoning and all is very high, right? [00:59:21] So it's very important that the human sort of comes in and approves things. But having said that, the primary use cases, so where, where an agent becomes, um, better than a rule-based system is if your problem statement, all of the user flow in itself is ambiguous, that you can't create rules, uh, and create a rule-based workflow, that's when an agent sort of comes in. [00:59:42] But agents interpretation of this ambiguity, uh, might not be very accurate as well. So in the near term, we think there would always be a human in the loop in the process. Uh, but long term, uh, we don't know, right? Five, five years down the line because we see new protocols [01:00:00] emerging. There's cps, there's a two A protocol. [01:00:03] So the world is, I think, looking at it. Point in time when agents would talk to each other and all of that. But even in those days, I think there are a lot of, uh, ethical sort of guardrails also would come into picture. So we'll have to wait and watch. So one very, uh, um, uh, sort of member model that I have is, uh, if you're familiar with this, so before, before joining here, I was in the banking and financial industry. [01:00:28] So I was building digital products for mutual funds and all of this. So I think this, this was in 2010 when there was this, uh, issue of flash crash in the us. I don't know if you're familiar with this. I, the stock market suddenly fell down by, I don't know, uh, 50, 60%. So essentially what happened that day is there was a bug when one of the autonomous trading platforms started selling a particular stock for pennies. [01:00:52] And there were other au automated platforms looking at the market, and they all went into panic mode. So these are [01:01:00] agents, they were not talking to one another, but they were, uh, they were making inference from a commonplace. And things went, uh, haywire, right? So that's an agent to agent conversation actually happening, even say 15 years back, right? [01:01:15] So when agents starts talking to each other, we will need a lot of guardrails in place. Um, so we'll have to watch and see how that, that would evolve. So what we currently have is just certain protocols on how agents could talk, right? So MCP is also essentially a protocol how a LLM client could easily interact with, uh, API, right? [01:01:36] Uh, so eventually how those things will evolve, we'll have to wait and watch. Uh, but we are super excited on what we see, uh, based on the initial traction from mia, uh, because a lot of marketers are now able to talk to these complex models in human language, and that, I believe is a good start. [01:01:55] Phil: Very cool. Iv, really appreciate your, your thoughts there. I got one last question for you. [01:01:59] How to Train Your Brain to Stay Content Without Chasing Wins --- [01:01:59] Phil: You're a founder, co-founder, CPO of a company. Obviously a team leader. Uh, you're also an avid reader. Uh, you post a lot about the books that you read. Uh, you give a couple of shout outs today. You're also a father, which I'm sure is keeping you busy. [01:02:12] Uh, I definitely empathize there. One question we ask everyone on the show is, how do you remain happy and successful in your career? How do you find balance between all the stuff you're working on at work while staying happy? [01:02:24] Rajeev: Uh, yeah. Uh, to, to start with the disclaimers. I, I don't want to make a claim that I figured out everything in life and all of that, but I primarily, uh, see contentment, right? So that's, um, that's I how I approach, uh, everything in life. Uh, so there's this very cool concept called, uh, hedonic treadmill. I don't know if you're familiar with, with this. [01:02:44] So, uh, so, uh, we all have. As per this concept, we all have a baseline level of happiness in us, and regardless of major life events, we would eventually regress back to that baseline level of happiness. So even if you [01:03:00] make a million dollars tomorrow, you'll be very happy, but eventually you'll happiness will come down to your baseline level. [01:03:04] Even if something very bad happens, uh, it'll come, come up to the baseline level eventually. Right? Uh, and, and we've been talking about MMM models and so MMM model also has something as a baseline, right? Which is even if you don't invest in any of your marketing, you still make your baseline, right? So all of us have this baseline level of happiness. [01:03:23] Uh, so, so the thing is, how could we. Increase that baseline. Right? Uh, so, so though I'm not a very religious person, I've, uh, read the bit in the eastern spiritual tradition, Hinduism and Buddhism and all of that, right? So one school of thought that is happiness is essentially a chemical reaction. So some of your neurotransmitters, like oxytocin, serotonin, and all fire in certain ways, and you feel happiness, but all these hormones are in your head today, right? [01:03:52] In your neuron stress. Like, can you just sit somewhere and make them fire and be happy without waiting for external stimuli, right? So I think containment is [01:04:00] the key to that. Um, uh, so Yeah, I, I'm not figured out everything else, but, but, uh, these are some of the interesting ideas that, that, uh, I have about happiness. [01:04:09] And one thing about having a daughter. uh, I, I, uh, so you also have a, a daughter? My, my daughter. Yeah. [01:04:16] So perhaps for every man out there, that's an easy hack. To increase your baseline happiness. So if you have a daughter, your baseline happiness increases, right? So that's an easy hack. It's not an easy hack, but it's a hack, right? [01:04:30] Phil: Yeah. Definitely not a, uh, always an easy hack. But yeah, I, I, I will say it has definitely increased my, my baseline happiness for sure. Love your answer. I feel like someone who is just checking out this part of the conversation is just like, oh, um, this, this guy must be a, a scientist or, or work with data to come up with that answer, but I love it. [01:04:51] Um, thank you so much for your time. Uh, Raj, this has been super fun. Um, obviously folks can check out, uh, Lifesight, uh, some of the stuff you're building is, is really [01:05:00] fascinating in this space. Anything you wanna plug? Anything exciting? You're, you're launching soon? Uh, this is gonna be dropping in in early July. [01:05:06] Uh, but yeah, feel free to plug, uh, some stuff. [01:05:09] Rajeev: Yeah. Uh, yeah. First before that, thank you, Phil, for this opportunity. I'm not much of a conversationalist, but you made me feel very comfortable. Uh, so, uh, and, um, uh, Yeah. [01:05:21] so, uh, at, at Lifesight, we offer free 30 to 45 day evaluation for everyone, right? So we can have a conversation with any brand. And if we think that this brand would benefit from a unified measurement program, uh, they could come in, they could test our system for 30 to 45 days. [01:05:41] It's free. Uh, we can show some value. Uh, there are different ways in which we can create model and show that the model has a good fit or not on your data and things like that. And after that, if they want to continue, that is a one year pilot program, right? Because this is not a one shot solution, this is an [01:06:00] itrate process. [01:06:00] We are essentially building a process for you, a framework for you, right? So there's a one year pilot program, uh, and as part of the pilot program, we would have a very detailed roadmap, sort of, uh, drawn up for, for the brand. And we will go through that. Uh, and the whole point is through that pilot program, we would train your marketers on this, um, uh, cost infr processes and incrementality system and all of that. [01:06:24] And eventually you could completely onboard Lifesight as your measurement tool. And then you can go ahead and build the rest of your business. You're not worry about measurements anymore. [01:06:35] Phil: Awesome. I appreciate that. Uh, folks gonna link out to the product and, and the site. Raj, this has been super fun. Thank you so much for your time. [01:06:43] Rajeev: Thank you.