Notebook LM podcasts
Auto Generated Transcript
===
Speaker 1: [00:00:00] Welcome to the Deep Dive. We are, we're really cutting straight through the complexity today.
Speaker 2: We really are,
Speaker 1: because we're diving into a, a really crucial high stakes issue in global development, measuring the value of partnerships.
Speaker 2: Mm-hmm.
Speaker 1: Our source material for this is a. Just a really comprehensive landscape review.
It was conducted by USAID's Momentum Initiative.
Speaker 2: Right. And that's a massive program. It's all focused on maternal, newborn, and child health, family planning.
Speaker 1: Mm-hmm.
Speaker 2: You know, the whole suite of reproductive health services. Globally.
Speaker 1: Exactly. So the stakes are incredibly high,
Speaker 2: and the complexity here is just immense.
I mean, these huge global challenges like making sure women and kids have access to critical healthcare.
Speaker 1: Mm-hmm.
Speaker 2: They increasingly rely on these diverse alliances, but
Speaker 1: proving that they actually add value.
Speaker 2: That's the problem. Proving that these alliances add value beyond what you know, one organization could do on its own.
Speaker 1: Hmm.
Speaker 2: That is notoriously difficult to capture. To quantify.
Speaker 1: So our mission today is to give you the shortcut. [00:01:00] We wanna help you understand exactly how major organizations are trying to monitor, evaluate, and learn from these absolutely critical alliances,
Speaker 2: right?
Speaker 1: We really need to unpack why measuring them is so, so frustratingly difficult, and what cutting edge tools are actually being used to capture their true value, that synergistic value.
Speaker 2: I think that's the key word right there. Synergy.
Speaker 1: Let's unpack this, and I think we have to start by first establishing what a partnership actually is in this context, because it's a lot more than just, you know, two people shaking hands.
Speaker 2: Absolutely. The foundational definition here is key. In this development field, a partnership is defined as a a dynamic relationship among really diverse actors.
Speaker 1: So we're talking private companies, NGOs, governments, the whole mix.
Speaker 2: Exactly. The whole mix. And they're all based on these mutually agreed objectives. Pursued through a shared understanding of, you know, their comparative advantages, who brings what to the table,
Speaker 1: A division of labor, essentially
Speaker 2: a division [00:02:00] of labor that, in theory maximizes everyone's strengths and the diversity of these alliances is.
Well, it's astounding. The review pointed to a huge spectrum of them.
Speaker 1: Like what?
Speaker 2: Everything from formal public-private partnerships or PPPs to cross-sectoral, intersectoral, these big multi-stakeholder engagements. It's not one single model. It's really a sliding scale.
Speaker 1: And the thing that struck me in the analysis was how clearly they laid out this partnership spectrum.
It categorizes the collaborations based on the depth of the commitment.
Speaker 2: Yes.
Speaker 1: Not just, you know how many people are in the room?
Speaker 2: Precisely. It starts at stage one, which they call leverage or exchange. This is pretty transactional.
Speaker 1: Okay.
Speaker 2: You're just delivering specific resources to achieve a shared objective.
It's basically training. Then you move up to stage two. Combine or integrate.
Speaker 1: So that's a bit deeper,
Speaker 2: a little deeper here. Partners are combining similar or maybe complimentary resources. The goal is just increased effectiveness or reach, maybe they, you know, merge their logistics networks to get supplies out [00:03:00] faster.
Speaker 1: Got it.
Speaker 2: But the real target, the absolute highest goal in this work is stage three. Systems transformation,
Speaker 1: systems transformation.
Speaker 2: This is the holy grail. Yeah. This stage involves partners bringing these essential complimentary resources together. Not just to solve a problem today, but to create, you know, structural systems level change.
Yeah. The kind of change that would be impossible for any single one of them to do alone.
Speaker 1: That idea stage three, it sounds a little abstract, so let's make it concrete. When we talk about momentum's partners, who are we actually talking about?
Speaker 2: We're talking about private healthcare providers, faith-based networks, huge multinational corporations, and crucially.
Non-health stakeholders.
Speaker 1: Ah, so like the water and sanitation sectors or education.
Speaker 2: Exactly. So an assistance transformation model. You're not just say, giving out health pamphlets,
Speaker 1: right.
Speaker 2: You might be working with the Ministry of Education to actually embed essential health services or curriculum directly into the [00:04:00] national school system.
Speaker 1: So you change the system forever, not just for the length of your project.
Speaker 2: That's the difference and that diversity. While it's essential for impact, it leads us directly into the core paradox of all of this.
Speaker 1: The measurement problem,
Speaker 2: the measurement problem, the review's. Single most important finding is that despite this global reliance on alliances, there is absolutely no standard approach, no set of standardized indicators to guide how people measure partnerships.
Speaker 1: Wow.
Speaker 2: Everyone is measuring something different.
Speaker 1: That's, frankly, that's terrifying. Given how much international funding is riding on these alliances, why do we even bother trying to measure them if it's so messy? What's the benefit?
Speaker 2: Well, you have to, measurement is necessary for two critical reasons. First, there's the internal reason.
It creates transparency. It builds trust among the partners, and it, you know, supports that common vision you need to keep going.
Speaker 1: Okay? So it keeps the partnership healthy.
Speaker 2: Right. And then second, there's the external reason, which is all about accountability. It's [00:05:00] how organizations determine the actual value of the partnership, the value add, and identify which approaches are successful enough to, you know, scale up.
Speaker 1: Okay. Here's where it gets really interesting for me, because despite those clear, very high level benefit. The sources show that most partnerships just, they default to the absolute easiest metrics they can find.
Speaker 2: They fall right into what the literature calls the measurement trap.
Speaker 1: The measurement trap,
Speaker 2: exactly.
'cause complexity is hard. Most of these alliances rely really heavily on easy to collect compliance or, you know, process indicators.
Speaker 1: So what does that mean in practice? What are they counting?
Speaker 2: They're counting the number of partners. Or how many times someone showed up for a meeting or whether a memorandum of Understanding was signed.
Speaker 1: Wait, wait. So we have. Billions of dollars and crucial health outcomes riding on whether people showed up to a Tuesday morning zoom call. I mean, that's just terrifyingly insufficient.
Speaker 2: It is. These process metrics, they're descriptive. Sure, but they tell you [00:06:00] nothing about the value of the partnership model itself.
Speaker 1: So why is it so complex? What's the real challenge here?
Speaker 2: Well, the source is detail a few reasons. Partnerships are by their nature unique. They evolve rapidly and their value is often really intangible. I mean, how do you put a number on increased trust? Or unexpected learning.
Speaker 1: You can't. And if you try to apply some standard metric, you just end up oversimplifying reality.
Trying to get a single definition of success to work across a dozen different organizations in different countries tackling different problems. It feels impossible.
Speaker 2: It is impossible. And. Attribution is another huge challenge. It's often really unclear how much the partnership specifically drove a result versus, you know, all the background work the individual organizations were already doing,
Speaker 1: right?
Speaker 2: And that challenge is what drives so many to just default to what's easiest to count.
Speaker 1: So if complexity is the enemy here. How do we pivot? How do we start capturing what actually matters?
Speaker 2: Well, the review synthesized these crucial success factors [00:07:00] that measurements should be trying to capture. These are things you often find in partnership checklists, like strong leadership commitment, a really clear shared understanding of the problem, and crucially valuing mutuality and cooperation between the organizations.
Speaker 1: So we know counting meetings isn't enough. We have to capture these squishier things like cooperation and shared vision, which brings us to the conceptual frameworks that are designed to do just that.
Speaker 2: This is where we start moving beyond just simple counting. The first big concept is partnership.
Synergy.
Speaker 1: Synergy again.
Speaker 2: This framework argues that synergy is the true mechanism for effectiveness. It's the ability to combine resources, skills, ideas to create a new outcome that is fundamentally more than the sum of the individual parts.
Speaker 1: So it's not one plus one, two, it's one plus one, and three. But how on earth do you show that?
Three? How do you prove it?
Speaker 2: You demonstrate it by mapping the process. The Synergy framework looks at the determinants, so things like available resources, partner characteristics, and then how that synergy is operationalized.
Speaker 1: [00:08:00] Operationalized,
Speaker 2: yeah. Through things like holistic thinking, communicating how different actions, address challenges, and then finally how that impacts partnership effectiveness and the ultimate project outcomes.
Speaker 1: So that three, that synergistic value. That could be a policy change or maybe gaining access to a community that neither partner could have reached on their own.
Speaker 2: That's a perfect example.
Speaker 1: It's not just more inputs, it's a completely different kind of solution.
Speaker 2: Exactly. A great example delivering, say 10,000 vaccines is one plus one, two.
Hmm. But if a health NGO partners with a logistics company and a Ministry of Finance to change the national cold chain procurement policy to guarantee future distribution stability,
Speaker 1: that's one plus one. Three. Three. That's the synergy.
Speaker 2: That is the synergy. Then there's another model, the causal chain for relationship outcomes.
This one connects the initial prerequisites and success factors,
Speaker 1: so the things, the partnership needs to be healthy,
Speaker 2: right? It connects those to partner performance and partnership practice, which measures things like organizational identity and [00:09:00] mutuality, and that all leads to the final outcomes of the partnership relationship.
Speaker 1: What's fascinating about that is it shows the internal health of the alliance, you know, whether they respect each other, if they communicate well, that it directly dictates the external results.
Speaker 2: Oh, it's a direct link.
Speaker 1: It's like measuring the quality of the foundation before you measure the height of the skyscraper.
Speaker 2: That's a great way to put it.
Speaker 1: So how do we translate these concepts, internal health and synergy into something tangible that we can actually track? We have to shift the whole goal from measuring effort to measuring value,
Speaker 2: right?
Speaker 1: And this is where two words become absolutely vital. Contribution and additionality.
Speaker 2: Yes, since perfect attribution. Proving the partnership was the only reason for an outcome is almost always impossible. In these complex environments, organizations have to pivot. They have to adopt a contribution approach,
Speaker 1: which means what exactly
Speaker 2: it means. You build a plausible evidence-based argument for how the partnership added value to the outcomes you're [00:10:00] seeing.
You're not proving it was the only cause. You're building a strong case.
Speaker 1: If we can't prove perfect causation, how do we stop organizations from just claiming credit for things that might have happened anyway, that's where Additionality comes in, right?
Speaker 2: That's the core of it. Additionality is the value gained that you can demonstrate would not have happened otherwise, or at least not at the same speed or scale.
Speaker 1: So you ask the question, did the partnership trigger investments that wouldn't have been made,
Speaker 2: or did it accelerate a necessary system change? By say five years, that speed, that scope, that structural change, that is the additionality.
Speaker 1: And because the standardized traditional indicators just fail to capture all of this complexity and synergy,
Speaker 2: the literature overwhelmingly recommends adopting mixed methods approaches and specifically something called complexity where monitoring or CAM methodologies,
Speaker 1: okay, this raises a really important question.
What do these CAM methodologies actually look like in practice? You can't just throw out all the numbers. So [00:11:00] how do they capture that, how and why that the quantitative data just misses?
Speaker 2: There are three critical CAM methodologies that help capture that specific value. The first one is called Most Significant Change, or MSC,
Speaker 1: most Significant Change.
Speaker 2: It's a purely qualitative storytelling approach. You literally harvest stories from partners, from communities, from beneficiaries, and you ask them. In your opinion, what was the most significant change that happened here and why was it significant to you?
Speaker 1: I love that because it completely bypasses the predetermined checklist.
It helps you identify those unexpected outcomes, the things you didn't even know to look for.
Speaker 2: Exactly.
Speaker 1: You might find out the most significant change wasn't the number of people you trained, but that the partnerships somehow unexpectedly enabled a local community health worker to get access to the regional governor for the first time.
Speaker 2: Exactly. That's the kind of thing MSC picks up. The second methodology is outcome mapping. This one tracked behavioral changes along the causal pathway to [00:12:00] impact.
Speaker 1: So instead of only looking at the very end result,
Speaker 2: right, instead of only measuring the final health outcome, it systematically assesses the influence the partners have.
On the immediate actions of what they call boundary partners, the individuals, groups, and organizations they interact with directly.
Speaker 1: Okay, that makes sense. And the third one,
Speaker 2: and finally we have social network analysis or SNA. This methodology actually maps and measures the relationships. The knowledge flows, the interactions between the people and organizations in the alliance.
Speaker 1: Oh, that's interesting.
Speaker 2: It's about seeing who the real linchpin is, the person everyone secretly goes to for information versus the person who just has the biggest title on paper.
Speaker 1: So you're literally measuring the health of the network itself. You can see who's playing a central role, who might be isolated, and where the bottlenecks in communication are.
Speaker 2: Exactly. It's so much richer than just counting if a meeting happened. You're measuring effectiveness based on actual influence and connection,
Speaker 1: and that all [00:13:00] moves us to a really practical point, which is the CDC guidance. The review referenced on developing a customized MLL plan. It seems to align perfectly with this whole participatory context aware approach.
Speaker 2: It does. A truly effective ML plan for a partnership has to be customized. The key steps are things like developing a specific theory of change. Just for the partnership itself,
Speaker 1: not just for the overall project.
Speaker 2: Not just for the project.
Speaker 1: Mm.
Speaker 2: And engaging all the partners in the design process. Choosing a manageable number of indicators.
And this is critical. Constantly documenting contextual factors,
Speaker 1: right?
Speaker 2: Because complex results are almost never transferable one-to-one.
Speaker 1: Building on that, the landscape review specifically recommended three standardized indicators for use across all the momentum awards. So these are the critical baseline data points,
Speaker 2: right?
Tracked along that causal chain. First, there's a compliance indicator, simply the number of partnerships supported, facilitated, or catalyzed by momentum.
Speaker 1: So basic context, who was involved? [00:14:00] What were their goals?
Speaker 2: It's the basics. Second is a process output indicator. The number of new solutions developed through partnerships to address problems prioritized by partnership members.
Speaker 1: Okay, so that's the first step towards showing contribution. It proves you actually generated something new,
Speaker 2: you're building the case. And the third, the outcome indicator is the number of changes as a direct or indirect result of partnership intervention. This is the critical piece that tries to connect the partnership's activity with better health outcomes.
Speaker 1: It's so vital to emphasize, right, that these three quantitative indicators are just the baseline. They're just the starting point.
Speaker 2: Absolutely.
Speaker 1: The review makes it explicit that these complexity aware monitoring approaches have to be used to add the depth to answer the how, the why, and the for whom these partnerships are actually driving outcomes.
Speaker 2: Absolutely. The review specifically recommends incorporating reflexive monitoring. Which is similar to what USAID calls a pause and reflect approach.
Speaker 1: So just stopping regularly to [00:15:00] see what's working,
Speaker 2: just stopping adjusting iteratively and making sure that internal communication is actually strong.
Speaker 1: And I imagine using MSC those significant change stories.
That would help validate that second indicator about new solutions.
Speaker 2: It's a perfect fit. And then at the end of a project, using something like contribution analysis or outcome mapping can really trace that plausible contribution and deepen your understanding of the incremental value that was created.
Speaker 1: So what does this all mean? It means effective partnership measurement requires a tailored toolkit, doesn't it? It needs a strategic mix of quantitative data, qualitative storytelling, network mapping, rigorous contribution analysis,
Speaker 2: right?
Speaker 1: Not just a generic spreadsheet that asks how many meetings you went to.
Speaker 2: That is the essence of all the findings. We moved from defining collaboration where the highest goal is that stage three systems transformation,
Speaker 1: right?
Speaker 2: To acknowledging the huge measurement hurdles in complexity and intangibles, and then finally recognizing that we have to shift [00:16:00] toward this custom complexity aware monitoring methodology.
It makes
Speaker 1: perfect sense.
Speaker 2: And if we connect this to the bigger picture, it's really about measuring what matters, not just what's easy to count. Knowledge is most valuable when you understand and apply it. And critical thinking about how we measure impact is just essential in a world that's overloaded with symbol, but often insufficient data.
Speaker 1: The review showed that those participatory approaches, you know, getting all the partners in a room to develop the indicators and interpret the results together are absolutely crucial for success. It becomes a collective learning exercise.
Speaker 2: It has to be.
Speaker 1: So here's a final thought for you to chew on. If that process of joint measurement is so essential for the health of the partnership itself, is the health of the alliance rooted in that mutual respect and that shared ownership, is that the most powerful and maybe non-quantifiable indicator of them all?
I.