Elie Hassenfeld: [00:00:00] Hey everyone, this is Elie Hassenfeld, GiveWell's co-founder and CEO. 

About a year ago, I sat down with Dilhan Perera from our new areas team to talk about the work we've been doing on family planning. At the time, it was a new area for GiveWell because we had only made one or two small grants. Since then, we've made a lot of progress. Over the past year, we've directed more than $20 million to family planning programs. Now, for context, that's a little bit less than 5% of the funding we've directed so far this year. So it's a real portfolio now, but it's still a small share of our overall work.

Family planning had been work that we were interested in for a long time, but foreign aid cuts created an even larger need. The US government had been one of the largest funders of family planning in the world. Their cuts were added to cuts from the UK and from the Netherlands, and so needs that previously existed got even worse. 

Now, [00:01:00] because we're just getting started here, a lot of what we're trying to do is to give in ways that can help us learn about the approaches we'll take in the future. And I think one way to conceptualize this is to think about family planning programs as needing three levels of work: commodity procurement, supply chain support, and then service delivery.

Commodity procurement means purchasing the actual goods that will eventually be delivered to people, and we believe, in many cases, there are just insufficient supplies of these needed health goods for the people who need them. Then the second is ensuring that supplies that are purchased actually make it all the way down the supply chain to the health facilities where people go to access services. And then finally, there's the service delivery itself. These could be, health clinicians or nonprofit workers who meet with clients, i.e., the people who need these services, and deliver them to the folks who need them, when they need them. So we think all of these areas are potential bottlenecks [00:02:00] to service delivery, and we're funding across this full spectrum so that we can learn more about where the needs are greatest.

And then the last thing before we dive in is, there's a lot we don't know in this space. And so, in some of the areas we support, the evidence is extremely strong that the types of programs we're supporting will lead to impact. And here, we have a lot of evidence that we'll describe in our conversation, but also there's a lot that we don't know, and we expect to learn more as we go. And I think this whole conversation will be a good illustration of the way in which GiveWell enters a new area, where we want to provide funding to help, but also, we'll be learning as we go about what works and what works best.

Dilhan, thanks for having this conversation with me. Just before we dive in, can you just introduce yourself and share a little bit about what you work on at GiveWell?

Dilhan Perera: Great. Yeah, thanks for having me, Elie. So my name is Dilhan Perera, I'm a senior program officer at GiveWell. I work on [00:03:00] our new areas team, which covers areas in global health and development that are new to GiveWell. So it's a team with quite a broad remit, and I've personally worked on a range of program areas, one of which is family planning that we'll be talking about today, but also things like HIV, tuberculosis, and medical oxygen.

Elie Hassenfeld: Well, we last talked about family planning about a year ago, and just to start off, it would be great if you could share an update on what you've done since then and what we've learned, and how you're thinking about the space now.

Dilhan Perera: Yeah, so I think at the time we last spoke, we had made one, possibly two, quite small grants in family planning, but we were actively investigating a broader set of areas. I think since then we've made a handful of grants that are now totaling roughly $20 million, so quite a big expansion in terms of amount of funding.

They've covered a range of areas within family planning. I think we've tried to be quite deliberate in covering different aspects of family planning that sort of fit together as a [00:04:00] portfolio and tell us about different bits of the system and help us learn about the area as a whole over time.

So they include, at the very upstream end, we made a grant to UNFPA Supplies, which is a UN agency focused on family planning and sexual and reproductive health, to procure family planning commodities, so the contraceptives, which are an important input into most family planning programs.

And then at the other end of the spectrum, we've also made a couple of grants that are more about family planning service delivery. So we made one grant to MSI Reproductive Choices, which is a large, international NGO focused on family planning to deliver a program called family planning mobile outreach in Nigeria and Sierra Leone. We've also made a grant to an organization called Lafiya, which is focused on increasing access to family planning in northern Nigeria by training up midwives and nurses and community [00:05:00] health workers to deliver family planning counseling and ensure they have the contraceptives to follow up with an offer of the products for people who want them. 

And then we've also funded some work in the middle. So there's the commodities and the service delivery, and there's a question of how the commodities get from the national warehouses to the health facilities and places where the services are delivered. And for that, we funded, again, MSI Reproductive Choices to support governments in three countries in sub-Saharan Africa to move the commodities from the warehouses to the facilities where there is demand.

And one of our earlier, much smaller grants was also on this part of the program, which is the supply chain of like, how do you get the commodities to the facilities? That was to an organization called Access to Medicines Initiative, AMI. They also focus in northern Nigeria, and they're doing various bits in one particular state to see what happens when you ensure that facilities aren't bottlenecked by the lack of [00:06:00] family planning commodities, seeing what happens to the provision of family planning services. And yeah, just through that work, understanding a bit more about what are the bottlenecks in the supply chains, which explain why these facilities didn't have the commodities in the first place.

Elie Hassenfeld: Something I wanted to ask about is you said that you made these grants deliberately to cover different aspects of family planning goods and service provision so that we could learn. What are the things that you're trying to learn? You know, how do you think these grants will help learning? And I presume, to the end of being able to make better family planning grants in the future, or build a larger portfolio of grants that are having impact. But just talk a little bit about the learning goals that you have with these grants.

Dilhan Perera: The big question for most of our family planning grants is, to what extent does it actually increase use of contraceptives, of things like contraceptive pills, and implants, and condoms by the people who have a demand or a need for family planning?

And I think it's not straightforward in most of [00:07:00] these cases that, you know, just the act of providing a contraceptive to a person actually increases their use, because in the absence of the program, they might have gotten that contraceptive through other channels. So through the private sector, for example, you can, you know, in a lot of places you can go to a pharmacy or a drug shop and buy some form of contraceptive. In some places there's multiple NGO partners working, so you might just be shifting supply from one NGO to another. There's also purely public sector supply. And so, just because an NGO is going out and to a community and delivering contraceptives there, it might be that some of those people would've made the effort and had the ability to get to a health facility, public health facility themselves to access the same thing.

So yeah, the impact of all of these programs on actual use of contraceptives is pretty uncertain, and that's a thing we're trying to learn about through all of these, particularly the more downstream service delivery grants.

Then there's an additional question, which is more [00:08:00] relevant for the commodity procurement grant, which is, to what extent do commodities that are procured and get into countries actually then make their way down to the health facilities where the services are delivered, and where people can access them?

You know, in some cases, the logistics that take the trucks and the people who take the products from the national warehouses to the facilities, they're just not there or there isn't the funding for them. So commodities do just sometimes get stuck in warehouses. In other cases, they might end up at facilities that have too much stock, rather than the facilities that need additional stock, so there's misallocation. And there's also questions of, you know, is there some leakage of products through the supply chain? Like, does it get diverted to places where it shouldn't be going, or do products somehow get stuck and then spoil, expire at various points?

So there's a bunch of reasons why we're not sure to what extent the commodities that are procured actually make their way to facilities. So yeah, I think they're the two big [00:09:00] categories of things we're trying to learn through these grants is, to what extent do commodities get to health facilities? To what extent is that the bottleneck for increasing supply? And then what difference does the more active service delivery make in terms of whether people are actually using contraceptives or not?

Elie Hassenfeld: Right. Those make sense. And then I guess there's also this big-picture question, which we talked about last time too, which is… family planning in many ways is different than a lot of GiveWell programs, because in a lot of GiveWell programs, if, for example, there's a child who wouldn't otherwise have gotten their routine immunizations, and then they get their routine immunizations, that is sort of self-evidently a good thing. And in family planning, there could be someone who, well, they may or may not want to have access to contraception at a particular moment. And so increasing use is not in and of itself good. It's only good if use is increased for people who want contraception at that moment in time. And so that's also, I imagine, like a very big question that we're asking, especially at that service delivery level.

Dilhan Perera: [00:10:00] Yeah, exactly. And I should mention with a couple of these grants, we've also funded separate independent research and monitoring to understand some of these questions. So not just to what extent these programs might be increasing usage, but to what extent the usage is wanted. So you know, to what extent are people making informed choices about whether or not they want to take up contraception? I think that is an important part of the thing that we're trying to aim for here is not just usage for its own sake, but usage because people want to be able to control when, and if they have children.

I will say, like an additional complexity here is that those preferences about whether or not to use modern contraception aren't fixed. So within a given person, they might change over time, and programs might have ways of increasing demand, for example, by providing more accurate information about the benefits and costs of contraception. So, it's not always straightforward to know who are the people who actually want to use contraception.

Elie Hassenfeld: And then another thing I want to just circle back on, because when we were speaking last year, it was, you know, still in the [00:11:00] relatively early days of government aid cuts. And I'm just curious, like, the extent to which aid cuts are a material part of the way that you're thinking about family planning programs today. So are they a major role, or is it, you know, sort of in the background, and you're just doing the thing that you normally do and looking for places where additional philanthropic dollars can do a lot of good, and this is one area where they could?

Dilhan Perera: Yeah, I think it's a bit of both. The foreign aid cuts have made this an area that is sort of… where there's more opportunities for cost-effective funding, I think. So, might have touched on this in the last time we spoke, but the US government was a big funder of family planning programs in general, and they were one of the biggest procurers of family planning commodities. So it's a bit unclear exactly what the status of US government funding that could be directed to family planning is at the moment. But they've stopped their procurement, at least we know that. So they were funding $60 million roughly per year of family planning products. That's not happening, so that's [00:12:00] increased the gap in funding for family planning products. And we know beyond that they were funding, $500 to $600 million worth per year of programming. We're not sure, still not sure exactly how that was split up. But yeah, I think things like service delivery, demand generation, also the supply chain, so moving products from warehouse to facilities, that was a big thing that USAID was funding, and some of that has resumed or continued, but I think family planning was an area that was specifically carved out of the supply chain programs that subsequently resumed. So, I think the need for the logistics grant that I mentioned was, pretty directly as a result of the foreign aid cuts.

But yeah, I think in general, all the things that we are funding are things that were to some extent funded by the big foreign aid organizations, and it's not just the US. We know that the UK is also, was a big family planning funder. They've announced aid cuts. The Netherlands was also another funder. So there's multiple foreign aid [00:13:00] cuts happening here that we think are affecting the space, and I think they just make all of the grants we've made here more impactful than they otherwise would have been.

Elie Hassenfeld: That's the big picture, and I want to now move to go through the grants individually. And so, you know, you laid out this very helpful framework of procurement, purchasing supplies; logistics, bringing the supplies from the place of, you know, delivery all the way down to like the health clinics or the place where the services are provided; and then the actual service delivery grants where you're interacting with individuals.

And I guess I'd like to talk through these grants in reverse order, starting at the direct service provision level. You mentioned two grants that we made, one to MSI and one to Lafiya. And so just explain to us what these organizations are doing.

Dilhan Perera: Yeah. So, MSI is a large NGO that runs a number of programs. The specific program we've funded is called mobile outreach, which is where a small team of trained providers, so it's I think typically two nurses and a driver, [00:14:00] go out in a four-wheel drive or some other vehicle with commodities in the vehicle, to specific communities or places that have been identified in collaboration with local stakeholders as being places where there's limited access to family planning services and products, and where there might be sort of unmet demand for family planning.

And they go to these communities, sort of spending one or two days at a time in each community. And then once they're there, they do a couple of different things, like one is a set-up at a prominent spot within a community. It might be a lower level health facility or just a more temporary site. And they deliver sort of group counseling, or information provision about family planning in general, why you might consider it, why it's important, the different options that people have.

And then following that, for whoever's interested, they can sort of queue up for a one-on-one counseling session where there, one of the nurses will ask the individual what their needs are for family planning, and then talk them through different options, and then ask [00:15:00] them to make a decision about, you know, do you wanna take up a specific method, or do you wanna go and think about it a bit more? And then typically, if someone's gone to that effort to line up, they will say that they want a specific option and that method is provided on the spot. 

What's unique about MSI is that their providers are trained to provide a wide range of different methods, so that ranges from short-term methods such as contraceptive pills and condoms, all the way to more long-acting methods. These are things like contraceptive implants and IUDs, which, you know, once they're inserted by a provider can provide somewhere between 3 to 10 years' worth of protection from pregnancy, where the individual would have the option to come back and get it removed if they want to start having children again. But if they don't, then they don't have to come back, and they don't have to worry about getting refills or anything for quite a while.

Elie Hassenfeld: You said that they're identifying areas with potentially limited access or unmet demand. What would that mean? Like, why would there be a place where there are people who are not being [00:16:00] reached but would like to have access to contraceptive products?

Dilhan Perera: Yeah. I think there's a number of reasons, and it can vary, obviously, depending on the place.

One is that there will be a health facility of some sort, a small health facility, which will have some types of contraceptive products. But for example, the specific health workers working in that facility just don't have the training to administer all types of methods. So in particular, the long-acting methods, which in some remote communities you can imagine there might be a preference for long-acting methods because they don't have frequent access, or it's harder for people to keep visiting a facility or a clinic. So administering those methods requires some clinical training, and either the providers just have never received the training, or they received it a long time ago and in practice aren't comfortable administering those methods.

I think another reason is that sometimes, either there's just not enough supply of the commodities in the system. So there might be a trained provider but not enough commodities at that specific health [00:17:00] facility to provide them. Or, there are commodities in the system, but the logistics to the more remote facilities hasn't been funded, and so the commodities just aren't reaching those places. I think another reason is that for some places, the nearest health facility is quite far away. And so, in these cases, MSI will just go out and you know, they'll have a sense of where these black spots are in terms of limited access to any sort of health facility, and they'll set up a tent or some other sort of temporary structure to actually just deliver the product, make sure there's a person there, and the product's there to provide the service.

Elie Hassenfeld: Right. And so there's just a lot of people who live in places that could be very far from a facility that would have the supplies they want in stock, could be far from a facility that even if it has the supplies in stock, has a health worker who knows how to implement the method that they prefer. And so MSI is just like… this program is intending to solve both of these problems by going to places where people have this limited access with trained [00:18:00] people who can deliver them and just bringing the supplies directly. 

I mean, so there's kind of this very like logical case that this kind of program would be effective. Is there empirical evidence showing that this program works? In the framing, you know, you raised this question of not knowing whether delivering supplies actually increases use. And so, I don't know, what do we know from this kind of program about its impact?

Dilhan Perera: Yeah. So I don't think there's very strong evidence about this particular type of program. There is more sort of descriptive case studies where you're looking at changes in contraceptive use over time and, you know, seeing whether that correlates with the places where MSI operates and where it doesn't. And there's some very suggestive evidence that is consistent with this program having an effect.

But there are relatively few RCTs, so randomized controlled trials, of family planning programs, and the more recent ones are not about this type of program in particular. We do know that there is an RCT currently ongoing about this specific type of program [00:19:00] implemented by MSI in Nigeria and DRC. So we're very eager to see what the results of that randomized controlled trial is. And we've tried to be very intentional about how can we collect data in this trial to have a sense about whether the results from that other RCT will generalize to the specific context and programs we're funding here.

Elie Hassenfeld: Yeah. So I want to come back in a second to the data that we're collecting and how we're evaluating this program. You know, I think it's interesting to think about the rationale for this grant with, I guess like, somewhat limited randomized controlled trial evidence for the particular program. I imagine it's something like the case is very straightforward in identifying places with limited access, and then bringing the supply and the clinician who can help, you know, deliver that product to the person who needs it. That's a very logical case where you'd expect to see increased use.

And so given the logical case, the fact that we can collect data [00:20:00] ourselves, the fact that there's an ongoing trial, it makes sense to be supporting this program now and seeing what we can learn, and then, updating as necessary based on the data that comes back. How are you thinking about that question of, you know, balancing directing funds to this program sooner in the absence of that strong empirical data versus, I don't know, waiting for more data to come in?

Dilhan Perera: So I think the way we typically do this, particularly on the new areas team, is to try to model out the cost-effectiveness, and through that process identify, like, what are the key inputs to that analysis that we don't have information on? Or conversely, you know, where do we have some information about that is informative, even if it's not a randomized controlled trial of the ultimate impacts of the program. 

So in this case, one thing that made me feel good about recommending this grant is that MSI does collect fairly detailed data about the people that it serves, so the clients that come and access contraceptives. And so they fund exit surveys of [00:21:00] clients to understand, who are the people that are being reached, what are their demographic characteristics? But also things like, had they ever used modern contraception before? Are they aware of other ways to access the specific product that they got, at the outreach?

And those indicators suggest that, in some places, 50% of the people that they're reaching have never used family planning before. So that's like, you know, a fairly good indicator that at least some of them wouldn't have used contraceptives in the absence of the program. And similarly, like 30, 40, 50%, depending on the context, will say that they're not aware of any other provider.

And these are surveys that are commissioned by MSI, it's not MSI staff who do the data collection, but there is a question of, you know, how reliable and accurate are these indicators? So we are, alongside this grant, funding a separate organization, IPA, Innovations for Poverty Action, to both replicate some of the types of data collection that MSI does themselves, so things like client exit surveys. But also then, go out and [00:22:00] do community-level surveys, so randomly sampling households in the areas that MSI is targeting to understand before the program, what percentage of women are using contraceptives, what percentage of people say they want to use contraceptives? And then seeing how that changes as the program is implemented.

Before we made this grant, we talked to MSI and we talked to other stakeholders to understand, you know, how does MSI identify the places that they target through the program, and what information is that based on? Does that seem like a reasonable process? So that's the sort of evidence that we gather to have some sense of, you know, this sounds good in theory, but, you know, is it likely to have impact in practice? And yeah, we're trying to be very deliberate about through the grants, you know, how can we collect more information beyond what MSI collects themselves to give us more confidence about that over time.

Elie Hassenfeld: And so like, similar, I'd say, that many of the grants that we make, it's a combination of triangulating information that helps the logical case or the theoretical case for the grant with empirical data from the [00:23:00] implementation itself, combined with independent, academic evidence about the program's impact over time in other locations. And then it's the process of putting that all together to make the decision here.

So that's MSI. I wanna talk about the other… Lafiya, which is the other grant where services are really being provided. So can you just tell us about that one too?

Dilhan Perera: Yeah, so Lafiya is a newer organization. They are scaling up very quickly, but to date, they focused on a handful of northern Nigerian states. Where MSI has their own staff who are sent in periodically to communities, so I think a community will get visited a couple of times a year, maybe four or five times a year, Lafiya tries to identify health workers, so nurses, midwives, community health workers, who are already living in communities with relatively low levels of modern contraceptive use, and then trains them up to provide counseling and specific methods, and also ensures that they have the products with them so that, for the people who are interested, they can provide the method on the spot.[00:24:00] And, you know, some of these health workers are not trained to deliver the full range of methods, so in those cases, they make sure that the women who want a different method are referred to a facility or another NGO, where, you know, Lafiya has some confidence that alternative methods can be provided. So it's sort of a more continuous version of service delivery that leverages people who are, you know, always in the communities where the service delivery is happening.

Elie Hassenfeld: And so it sounds like they're trying to hit one of the particular bottlenecks that you described previously, which was a situation where people come to health facilities or the you know, even it's a low-level facility, facility has the product in stock, but the health worker is not trained to deliver that method. If they can increase training, then that's a potentially very low-cost, high-leverage way of increasing use, because they're just taking the existing capacity, both the health worker and the commodity, and enabling them to serve a greater number of people more successfully. [00:25:00]

Dilhan Perera: Yeah, that's right.

Elie Hassenfeld: So tell us more, like what do we know about... you said it's a newer organization, so I don't know, I kind of always have the same sort of questions, which is like, well, what do we know about how effective this program is? Like on its face it's very attractive, because presumably it's lower cost, like they don't have to send vehicles out and pay for additional staff. It's like they're just leveraging the existing resources.

But then it's, you know, it's harder to affect change with training and to measure the impact that that kind of work is having. So tell us more about, you know, that program, sort of both what we know now and, you know, why we see this as a promising program to support. And then also, what we'll be learning and how we'll be tracking the success of this grant over time.

Dilhan Perera: Yeah. So, one thing that gave us confidence to provide this grant is that Lafiya themselves are very oriented around learning about their impact, and so they think a lot about their M&E. And what we know from their M&E, their monitoring and evaluation, so far is mainly the number of people who are being reached, and [00:26:00] both the number of counseling sessions being provided, and of those, how many women say they want to use a method and which method they say they want to use. And also, where the service delivery occurs. So in some cases, it's within a facility, so someone might come in for some other reason, there's a trained worker there, and they'll be offered family planning counseling on the side. Whereas in other cases it's the nurse or community health worker going out to a spot in a community similar to MSI or going door to door and trying to find people who might be interested.

And so, we don't have as much information in this case about exactly who is being reached and how likely is it that they wouldn't have used contraception in the absence of this program. I think Lafiya does track, you know, how many of the people being reached have never used family planning before, and I think in Northern Nigeria, it tends to be around 50%, so pretty similar to what we hear about from MSI. I should say Northern Nigeria is one of the places in the world with the lowest levels of current use of modern contraceptives. [00:27:00] So I think that's why a lot of funding is focused there is to try to increase that to levels that are similar to what you see in other places.

Because this was a smaller grant, we're not funding independent monitoring alongside this grant, but we do expect Lafiya to report out frequently about what they're seeing through their own M&E systems, and we've also given them some steer about where their M&E might not be giving the full picture, and I think they're very keen to sort of strengthen their M&E going forward.

So for example, the number of counseling sessions that occur, I think is self-reported by the people who are delivering the services. So, you know, it's possible that it could be inflated. Lafiya is very aware of this risk and does data quality audits, so they listen in to the surveys that are recorded as part of this to make sure they actually happened, and they do sensible things on this. But I think, yeah, there are ways that the M&E could be strengthened, and I think they're very aware of that and are working on that.

So they've been a very transparent grantee to us, and I think that's both valuable in us getting a better handle about what the actual impact is over [00:28:00] time, but in terms of understanding the broader family planning space, and where they think there are bottlenecks versus not.

Elie Hassenfeld: Yeah, and I think one distinction that's worth drawing out between these two grants is just the magnitude of the funding we're providing in each case.

And so MSI is much closer to… they're a large, established organization that has done broadly this kind of work for a long time. And we provided them a little bit more than $6 million for the work that we're describing here. And then Lafiya is, like, a younger organization that we think has the potential to do a lot, and we provided them a million dollars, so about, you know, one one-sixth the size.

And then I imagine that we'll be watching both closely, but obviously, like, part of the hope with an organization like Lafiya that is growing, and, you know, we've seen this trajectory with organizations we've supported in the past, like New Incentives, where they start smaller and then part of the process is them, you know, building up their own track record and improving their monitoring and evaluation so that they [00:29:00] can grow and become more effective over time.

Dilhan Perera: Yeah. And I will say that also I should mention, Lafiya is very keen to have more evaluations, rigorous evaluations on their own program impact. They've already commissioned a, what's called a difference-in-difference study, which provided some suggestive evidence that they were having an effect, not just on contraceptive use, but also unintended pregnancies, which is, you know, one of the main downstream impacts of increased contraceptive use.

And they're looking for other opportunities to sort of run rigorous evaluations or have other people commission rigorous evaluations about themselves. So yeah, I think they're very open to learning and understanding what the impact is. And our funding is supporting them in that growth journey and helping them both scale up, while also better understanding whether they're actually having the impact that they think they are.

Elie Hassenfeld: Cool. Well, that's really great. 

So that's the sort of end of the delivery chain, when providers are interacting directly with clients and health workers, and let's go all the way to the other end of the spectrum and talk about procurement. You know, we made a grant to the UNFPA to procure family planning [00:30:00] commodities. Tell me more about that.

Dilhan Perera: Yeah. So, there's a part of UNFPA called UNFPA Supplies. So we provided an unrestricted grant to UNFPA Supplies to do their work. I think broadly speaking, their work involves pooling funding for family planning commodity procurement from lots of different funders, and then working across lots of different countries. I think it's something like 50 or 60 different low- and middle-income countries to understand, what's the total need for family planning commodities in the public sectors of each of those countries, how much of that is already being funded, what the gap is, and trying to allocate the pooled funding and the commodities across all these different countries in a way that's roughly proportionate to what they estimate the need to be.

And then they have agreements with the manufacturers of the products where they've negotiated prices. And so they buy the products from the manufacturers. They sort out the international freight to get them from the manufacturers to the national warehouses. And then they also do some [00:31:00] work with governments to try to ensure that there's some domestic financing for these products. So it's not just purely donors funding the products, but over time at least, where governments have capacity, that more and more of the financing is being provided locally rather than through donors.

Elie Hassenfeld: Just like kind of walk me through the case where we must believe that overall, insufficient supplies are a major bottleneck to access, and I know we have some question about the extent to which supplies actually, like, make it all the way down the chain to reaching the people who need them. But we must believe that overall, a large proportion of those supplies must make it there. And so just like walk me through how we answered those two important questions about this grant.

Dilhan Perera: So from the very beginning of our exploration of family planning, we, you know, usually with a new area, we make a deliberate effort to go out and talk to lots of different people who have more experience with the space. We consistently heard from almost all stakeholders that one of the major reasons contraceptive use, you know, wasn't as high as [00:32:00] it could be given demand is that there's just not enough funding for the actual product. 

So there's stock-outs in facilities. Sometimes that might be because of logistical issues within a country, but often it's just that there's not enough funding for the commodities in a country. We heard that funding gaps for the products was a chronic issue, even prior to the foreign aid cuts. And then in light of the foreign aid cuts, it became apparent that those chronic underfunding had been exacerbated, because USAID was the second biggest procurer of family planning products for the public sectors in low- and middle income countries. Yeah, so we definitely think that there's a gap here. We're not sure if it's the main bottleneck or the only bottleneck. So, filling that gap in and of itself might not be enough to increase uptake to the point where it could be. But in most places, we think it is one of the bottlenecks.

The other part of the reason we made this grant is that the commodities themselves are relatively cheap. We think roughly it costs something like $10 in total, the all-in cost to deliver a year of modern contraception to someone who wants it, including the service delivery cost. Of that, [00:33:00] $10, $2 or $3 is the commodity. So I think part of the reason here is that in places where the commodity is the main bottleneck, for $2 or $3 worth of donor funding, you're getting potentially a whole year of contraceptive access that otherwise wouldn't have occurred, which we think is worth a lot more than a year when you consider the health benefits, the economic benefits, and the sort of wellbeing benefits for people who want to, you know, have some control over when they have kids.

Even if some portion of the commodities don't actually make it to health facilities, even if some of the uptake is just substituting for uptake that would've occurred through other channels, we think this is still probably very, very cost-effective just because of how cheap the commodities themselves are relative to what we see as the benefits of having access to modern contraception.

Elie Hassenfeld: And then how are we going to learn about how this grant went? So in the future, what will we know? Because it makes sense that the way you're talking about this is that it's likely that [00:34:00] one obstacle is insufficient commodities. We want to provide some funding for commodities that can help us learn about the extent to which this is a bottleneck in certain locations. And so then I'd love to hear more about like how we will determine that in the future.

Dilhan Perera: Yeah. So I think because this is a more upstream grant, it will be harder to learn about over time, or be less straightforward to learn about. But we are actively shaping the monitoring and evaluation that will happen through this.

So I think one part of this is that UNFPA themselves are actively strengthening the extent to which they can track the commodities that they buy through various parts of both the international and local supply chain, you know, what percentage of products procured actually make it to national warehouses, and then from there, what percentage make it down the chain to facilities?

Yeah, I will say that the sort of within country bit is not the main focus of UNFPA Supplies. They mainly focus on getting the products into countries. So to complement the work that UNFPA is doing, we are considering funding [00:35:00] work in a couple of countries where we expect a relatively large portion of their commodities to go. Both to track the supply chain, so to what extent do products make their way down from warehouses to facilities. And then at the facility level, how does, you know, delivery of UNFPA Supply shipment, to the extent that it can be attributed, how does that affect stock levels at a facility? How does that affect the number of family planning services that are required at the facility? And then potentially also doing some sort of exit surveys or interviews with the women at these facilities to understand, for example, if they did access a product at the facility, what would they have done if they tried to do that and turned out that the product wasn't there? So trying to get some sense of, what is the counterfactual in the absence of the products being available at the facilities?

We're still sort of figuring out that independent M&E because it's not straightforward often. Sometimes it's not possible to distinguish between which products were procured by UNFPA versus through other funders or other mechanisms. So we're [00:36:00] still a bit unsure about what's possible here, but we do want to both track the supply chain, learn more about the supply chain within a country, and then at the health facility level, understanding what the impacts of having more stock is for both the facility service delivery and the people who end up getting those services.

Elie Hassenfeld: I don’t know if you can answer this question, but, you know, GiveWell does a lot of support of programs that are, in one way or another, supporting commodity purchase and delivery. So the paradigmatic case would be malaria nets. And in the case where we support the delivery of malaria nets, we're supporting either the Against Malaria Foundation or Malaria Consortium to deliver nets in a particular location. You know, and that can be like a sub-national location often. 

In this case, we're talking about providing support to, like this massive multinational commodity procurer. And I'm just curious, like why do you think in this case we're [00:37:00] supporting a vehicle that is this multinational commodity procurer? In other cases, we do this differently. Were there options to do this a different way? Would you prefer to have done it a different way if you could? Or is there a reason that this is the optimal way within family planning?

Dilhan Perera: Yeah, I think we're not sure if it's the optimal way. I think the MSI outreach grant that I mentioned before is closer to what we've typically funded. So we know where they're targeting, we know that they're taking the services there. In some cases in some of the places, they are also doing the commodity procurement themselves, so they're bringing the commodities. In other cases, they're accessing commodities that are available because UNFPA and others have procured the commodities for the government, and then the government has granted those commodities to MSI to deliver.

I think the main reason we were interested in making this sort of more upstream pooled procurement grants is that the ability to have impact at a much larger scale is just a lot larger. So if it is working well, then it's quite an efficient and effective way for us to have impact. The grant we're making comes nowhere [00:38:00] close to closing what UNFPA thinks their funding gap is, which in itself is based on funding gaps that each country estimates they have. So if we felt more confident about that over time, then we could fairly easily make a decision to direct more funding to that and have much more impact.

Maybe family planning is another area where we think that, at least in a lot of places, the facilities are there, the infrastructure to deliver the services is there, some people have been trained, and often the gap is just the commodities. And we think the most effective way to identify those places is through organizations like UNFPA who are already engaging with governments. Governments in turn are engaging with regions and local governments and facilities to sort of understand what the needs are. So rather than us individually going and trying to map out exactly where the gaps are, you know, to some extent delegating that to a system or a mechanism that is already doing a lot of that work makes sense. And we just think because of that leverage, because a lot of the other parts of the system might be in place and the commodities themselves might be the thing that's [00:39:00] missing, there's just a lot of leverage here to have impact quite cheaply.

Elie Hassenfeld: Well, that all makes sense.

So, we talked about the end of the chain, the beginning of the chain. Now let's just take a few minutes and talk about the middle of the chain, the support we've directed to help improve logistics and the flow of the supplies through the system. So tell me more about what we did there.

Dilhan Perera: Yeah. So this is just a one-year grant to, again, MSI, that we think is sort of a short-term solution. Which is that, like I mentioned before, the US government was previously funding supply chains in a lot of countries, and the family planning elements of those supply chains in a lot of places have been cut. And there just either hasn't been a coordinated response, in terms of who is going to replace that, or it's very fragmented, so like different people are working at different times, and there's often gaps that emerge because the sort of replacement isn't coordinated.

This is sort of a grant we made reactively because MSI, through their existing work, noted that there were challenges in getting the commodities from the warehouses to the facilities. We had heard about these gaps for a long time. Again, some of these were [00:40:00] chronic, some of these were made worse by the USAID withdrawal. We asked MSI to identify which of the countries where they thought there might be gaps, you know, had the most acute need. And they identified three countries, which were Zambia, DRC, and Malawi, where they were both aware of gaps for funding for this through their conversations with governments and through their first-hand experience at health facilities, and where they had the teams in place to help fill that gap.

So what MSI are doing with our funding is basically using their existing staff and their existing vehicles to find out where the gaps are, and literally taking the commodities from one place to another. And in some cases where they think there's, you know, just a lack of infrastructure to do the work, they're buying additional facilities or making upgrades to warehouses to ensure that the commodities can be stored well. Literally, you know, working hand-in-hand, side-by-side with government workers to strengthen bits of the logistics system that where there just aren't enough resources at the moment.

Elie Hassenfeld: And what do you think we'll be able to learn about how this went? [00:41:00] Like this is another case where the grant makes a lot of conceptual sense. It's that you have supplies that come into the system, they need to go somewhere far out. You know, part of, in looking at the grant materials, something that we did was even provide recently, like a little bit of increased funding because fuel costs have gone up. I mean, the costs of distributing supplies like literally have these very concrete costs that need to be paid for. And so, if you can pay for supplies to be distributed that are not going to be distributed otherwise seems very effective. And then it also is very hard to track. So like what we'll be able to know about the impact that this kind of grant has had?

Dilhan Perera: Yeah. One thing that's interesting about this grant is that MSI will be reporting to us both what they know from their own M&E, so you know how many commodities they're actually delivering using their trucks, but then also giving us snapshots of what they know from the governments' logistics and health information systems. So I think there's a benefit there of just seeing what those indicators are and how they change over time. 

It'll be very hard to [00:42:00] attribute any changes to this specific logistics work, so we're not really expecting to get a definitive sense of the impact through that. But, we will see, for example, if some of these existing systems suggest that there are stockouts persisting despite this, then we might think actually there was some other bottleneck, or this wasn't having the impact that we thought.

At the same time, I think there's just a more like meta benefit for us of being aware of what data already is being collected, you know, what the indicators are, what we can learn from that, and what we can't, so that hopefully if we consider a similar grant in the future, we'll be better placed to sort of more proactively set up the evidence generation to get a better sense of what impact it's having.

Elie Hassenfeld: Great. well, thanks, Dilhan. This has all been really interesting. It's like an amazing new piece of our work, and it's really exciting for me to see how much progress we've made, and you've helped us make over the past year since we talked last. You know, really a lot of progress, and I'm glad that the work we're doing will hopefully enable a lot of people who want contraceptive services who can't get them today to get more of them in the future.
--

Elie Hassenfeld: [00:43:00] Hey, everyone, it's Elie again. 

So one important takeaway from this whole conversation is that the giving we're doing here is built not only to provide help in the short term, which we believe it will, but also to teach us about this area, because all of the grants we've discussed come with questions that we still need to answer. Does providing commodities actually increase use, or does it substitute for what people would've gotten another way? Do the commodities we help procure actually reach facilities, or do they get stuck or expire before getting there? And maybe most importantly, when use goes up, is that what people genuinely want? Are people making informed voluntary choices about the family planning and the contraceptive methods that they're undertaking? In several cases, we funded independent research and monitoring precisely to help us answer these questions well.

And so, you know, all of this work is still early days for us, but it's really [00:44:00] exciting for me to see how much progress that we've made over the past year, and I hope that all of you will stick with us as this work continues to grow and we learn more so that we can share back with you. We see a lot of potential for this area to become a much bigger part of our work in the future, and that will be built on the results that we're able to get from the grants we've made so far.

Thanks as always for your interest in GiveWell and for your support. We really appreciate it.