GiveWell Conversations

Routine childhood immunizations, provided on a set schedule over a child’s first two years of life, are very effective at reducing some of the major causes of death in young children. We estimate that in the provinces where we’ve supported programs in the Democratic Republic of the Congo (DRC), about one in 13 unvaccinated children dies from vaccine-preventable diseases before the age of five, yet only about half of children there receive the vaccinations they need. 

Much of our Vaccination research subteam’s work focuses on finding ways to make it easier for caregivers to have their children vaccinated, such as bringing vaccines closer to where parents and infants live. This is especially important in places like DRC, where coverage is low and disease burden is high. Following a request for proposals and a request for information seeking targeted vaccination outreach programs, we funded 11 additional programs, run by distinct organizations—six of which will be implemented in DRC. This portfolio builds on years of investment in our research team, which allowed us to evaluate more than 100 proposals and fund several programs side by side to learn which approaches work best. 

In this episode, GiveWell co-founder and CEO Elie Hassenfeld speaks with Senior Program Officer Natalie Crispin, who leads GiveWell’s vaccination grantmaking, about the programs we funded in DRC. They discuss barriers preventing many children in DRC from getting vaccinated, how the programs we’re funding aim to reach them, and what we hope to learn.

Elie and Natalie discuss:
  • Identifying barriers to full vaccination in DRC: Based on the evidence we’ve reviewed, our past experience funding vaccination programs, and the experts we’ve consulted, we think reliable access to opportunities for children to be vaccinated is a major barrier to coverage. The typical immunization schedule requires eight or nine visits to a health facility over a child’s first 18 months. In DRC, families living within about three miles of a health facility are generally expected to travel there. This can mean walking several miles to the nearest health facility while carrying a small child, with no guarantee that the needed supplies or staff will be available to vaccinate their child when they arrive. Mobile vaccinators travel to reach families living further than three miles from a facility, but there are often insufficient resources to ensure these outreach sessions take place on the needed schedule and in the hardest-to-reach communities. 
  • Funding different strategies for reaching under-vaccinated children: Each of the six outreach programs we are funding in DRC includes two core components: (1) a method for identifying the communities that need additional support and (2) a strategy for reaching those communities, often by increasing how often vaccination teams travel from health facilities to underserved villages. In many of the programs, a small vaccination team travels by motorbike with a cooler of vaccines, sets up in a central spot that community health workers often announce in advance, and gives each child the vaccines they need. The details of the programs vary. For example, one organization is implementing an app that helps health teams log their work, allowing supervisors to track activity and make changes in an effort to increase coverage rates. 
  • Learning which approaches work best: In order to compare the cost-effectiveness of various approaches and organization types, we've chosen a diverse grant portfolio of programs and organizations (including large international NGOs, organizations focused on data and technology, and nonprofits with deep experience with the DRC health system). To assess the results, we’re funding independent coverage surveys before program implementation and every year afterward, using comparison groups wherever we can to assess whether changes in coverage were caused by the program. We’re also drawing on programs’ own monitoring data and talking with government officials, other funders, and implementers in the same areas. Over time, we plan to scale up those that increase coverage most cost-effectively to help people in need as much as we can. 

We expect efforts to increase coverage of routine childhood immunization to make up at least half of our vaccination grantmaking, as these vaccines have the greatest potential to save lives. The team is also assessing the cost-effectiveness of filling funding gaps for purchasing vaccines, starting with malaria vaccines, and researching other vaccines that aren’t fully funded. With a larger, more experienced team, we can now research vaccines in more depth and fund multiple vaccination outreach programs to assess which are most cost-effective. We’re bringing that approach to all of our work, so we can do the most good we can with donors’ funds.

For more on how our vaccination grantmaking has evolved, see our earlier podcast episode with Natalie. Visit our All Grants Fund page to learn more about how you can support this work, and listen or subscribe to our podcast for our latest updates.

This episode was recorded on October 27, 2026, and represents our best understanding at that time.

What is GiveWell Conversations?

Welcome to GiveWell’s podcast sharing the latest updates on our work. Tune in for conversations with GiveWell staff members discussing current priorities of our Research team and recent developments in the global health landscape.

Elie Hassenfeld: [00:00:00] Hi, everyone. This is Elie Hassenfeld, GiveWell's co-founder and CEO, and today I'm going to talk to Natalie Crispin about our portfolio of work focused on increasing vaccination coverage in low-income countries. It's really fun for me because Natalie is GiveWell's second longest tenured employee after me, and so we share a lot of history at GiveWell and context that we're going to use in going through this conversation.

Vaccinations are a really core public health commodity and low coverage, in some ways, seems like a basic problem. There's a lot of money that goes to support immunizations globally, but notwithstanding all the funding that goes into supporting vaccination, there are still very wide gaps. On average in DRC, the country that we'll be talking about today, only about 50% of needed vaccinations are received by children.

As we get into this, one of the things that you'll see [00:01:00] is that while in some ways vaccination is simple in the scope of the programs GiveWell supports, because once the child receives its vaccination, that incurs some health benefit, but it's still very complicated to get it right and ensure that kids received all the vaccines that they need.

In this case, we'll be talking about a series of experiments that we're running to figure out how we can go about increasing vaccination coverage in areas that currently have relatively low coverage. We're hoping that by funding a series of organizations in different locations, we can learn more about which approaches work best, and then take those learnings to support increased coverage in a larger area.

For the vaccines team as a whole, this series of experiments is just one part of what we're doing, but it's what we're going to focus on in this conversation. And one thing that I think is particularly exciting is that as new vaccines are developed, we add them into the approach we have to delivering vaccines to kids. And so we'll talk about [00:02:00] some relatively newer vaccines, like the malaria vaccine, that will benefit from improved approaches to increasing coverage. And, as I hope new vaccines are developed in the future, hopefully they will also benefit from some of the learning that we have about how to deliver more vaccines more effectively.

I'm gonna talk to Natalie today, but the work that we're discussing was supported by a larger group of people on our vaccines team, so thanks to everyone at GiveWell for their work on this, and Natalie, just excited to get into this conversation with you. Thanks so much for joining me today.

Natalie Crispin: Hey, Elie. Great to be here.

Elie Hassenfeld: Tell us a little bit about vaccination, and what you're focused on, and why it's so important to be working on this problem.

Natalie Crispin: Yes. So we are a few years in now to having a vaccination team at GiveWell and a strategy that is trying to look across the vaccination space and understand where the major gaps are, where GiveWell can be additive to the space.

We have broken our [00:03:00] work down into two main buckets for now. One is the piece that we'll talk more about today, which is around increasing coverage of routine immunizations. This is the set of vaccines that are delivered to young children, generally those under two, and cover a variety of diseases. Our work there is looking for particularly low-coverage, high-disease-burden areas, and providing support for those systems to strengthen their delivery of vaccination services.

The other bucket is around vaccine procurement, so ensuring that governments have the vaccines needed to deliver both through these young child routine immunization systems, and also sometimes via additional mechanisms to other age groups. That work is newer for us. We haven't done as much in that so [00:04:00] far.

Elie Hassenfeld: So we're going to talk a lot about increasing coverage of routine immunizations in low-coverage, high-disease-burden areas. Give me a sense of the scale of the problem, how many children aren't receiving the immunizations they need?

Natalie Crispin: There's quite a variety across countries of levels of vaccination coverage. So in some countries, including those that are very low resource, they can have quite high coverage rates in 70%, 80%, even 90% across most doses. And then there are systems that have been weaker at delivering routine immunizations to young children. This includes some quite large countries that have large populations of young children.

Some of the ones that we're focused on right now include DRC, Nigeria, Somalia, South Sudan, Chad, Ethiopia. We've made grants in all of those. We think that there's a particularly high burden of unvaccinated children [00:05:00] in DRC and high disease burden there. Across DRC, there's something like 50% coverage. That will vary a lot by province within DRC, as well as by which vaccine we're talking about, which dose in the series. So that's a very sort of crude average. But there's a lot of room for improvement in vaccination coverage in some of these high-population countries, and DRC has become a real focus because of that.

Elie Hassenfeld: So we're going to talk about DRC. Can you just tell me a little bit more? Like, what does 50% coverage mean? And maybe, just because the vaccine schedule is sort of complicated, it's helpful just to walk through the primary vaccines included in routine immunizations, and what it means for there to be on average about 50% coverage.

Natalie Crispin: Right. So a child needs to have an interaction with the health system and receive vaccinations something like eight times across their [00:06:00] first about 18 months of life. A lot of that is concentrated in the very young age range. You tend to get higher coverage for the vaccines that are delivered soon after birth, and so some of the vaccines that are delivered near birth or in the first few weeks include the BCG vaccine for tuberculosis. The Penta vaccine; penta for five vaccines combined into one that includes, pertussis, tetanus, diphtheria, and a few others. And polio vaccines are delivered at the same times. Those tend to be fairly good coverage, though you do need three doses of the Penta vaccine, and those are delivered alongside the PCV vaccine, which is a vaccine that targets one cause of pneumonia. So you do see dropout through that series.

And then we have malaria vaccines, which are new in the last couple of years, and those are delivered, depending on the [00:07:00] country, starting at five or six months, and then with at least one month in between additional doses. And there you're trying to get four doses, three in the primary series in the first year of life, and one in the second year of life. And so there are many touch points that need to happen to get full protection, and it's easy for a child to just drop out at some point in that series, because it's a big load for the caregivers who are bringing those children into health facilities to do that so many times.

Elie Hassenfeld: And then for these vaccines where you need multiple doses, do we know what proportion of the benefit comes from some subset of the doses versus finishing the full schedule?

Natalie Crispin: Yes, it differs by vaccine. So for example, the measles vaccine, which is given at around nine months typically, gives quite high protection from one dose. But a second dose is also recommended in the second year of life. And then there's the Penta [00:08:00] and PCV vaccines where we think there's pretty good protection from two doses, but three doses is the recommendation helpful for achieving strong protection. Malaria vaccine is a particularly demanding schedule where the evidence suggests that there's quite low protection given from two doses and even less from one dose, and the third dose is really where you start to see significant protection. And then that protection does wane fairly quickly, such that a fourth booster dose in the second year of life is very helpful as well.

Elie Hassenfeld: And how much do we know about the ongoing increased mortality, increased child deaths that come from the lack of fulfilling the vaccination schedule? Meaning, do we have an estimate or a sense for if we were able to improve coverage from, you shared an average of about 50%, to something materially higher, what would that mean for child mortality?

Natalie Crispin: Together [00:09:00] these vaccines cover a set of diseases that are a real major driver of mortality. So, malaria vaccines are a sizable chunk, something like a quarter to, in the highest burden areas, up to a half of the value of delivering this set of vaccines in terms of disease burden. And malaria is one of the biggest drivers of mortality in the places where vaccines are used.

However, that still leaves 50% to 70% or so of the burden of disease that is reduced through vaccination programs as this other set of diseases. So, pneumococcal disease, which PCV protects against. Pneumonia is a very major killer of young children in low- and middle-income countries. Rotavirus is a vaccine-preventable disease that is a major cause of diarrheal disease, which is another large killer of young [00:10:00] children. In some places, measles is a major cause of mortality. Pertussis can be as well. So those are the ones that, when you increase coverage above current levels, what you see in terms of reduced disease burden.

Elie Hassenfeld: So you've shared a little bit about the vaccination schedule. There's a number of doses over a period of time where caregivers need to bring their children to receive the doses. Right now, as you think about trying to address this problem, which is essentially trying to, using DRC as an example, increase coverage of immunizations from around 50% to something materially higher, what are the main causes of the gaps in coverage that you think we need to try to overcome?

Natalie Crispin: Our read of the evidence and from a bunch of conversations with experts on vaccine programs, and specifically in DRC, is that access to services—so having services that are reliably available within a distance [00:11:00] that caregivers can reach—is a really major barrier. Now, increasing access might not be able to get you to 100% coverage or particularly close to that, but at the current margin, we think that that's the barrier that if you can alleviate that, you could get a really big increase in coverage. And so that's where we've decided to focus for now and try to learn more over time about what are the other barriers that these programs could help alleviate.

Elie Hassenfeld: What do you mean access to services? What's the status quo, especially in places where kids are under-vaccinated, and then what kinds of interventions would lead to improved access to services?

Natalie Crispin: I think some of the biggest barriers are reliable vaccination sessions in health facilities. With routine vaccination, the primary method across really all countries is that caregivers are bringing their children to health [00:12:00] facilities, and that's because it's on this age-based schedule. For vaccination, you're really hoping to get children at six weeks, 10 weeks, 14 weeks of age, et cetera. So it's worked really well to build up this system in which people come to the health facilities when their child is the right age.

In DRC, in many places, the health facilities are far from where people live. The roads and the transportation systems are quite weak. There's cases where people may show up and not receive services because they aren't being provided that day—the staff haven't arrived, or the supplies haven't arrived. And so, step one in many cases is to ensure that regularly scheduled services with the staff and supplies needed are happening. That will help those who are within a reasonable distance of health facilities, and [00:13:00] once they know that reliable services are available, they will make those trips.

In addition, we are looking to increase the frequency of outreach or mobile services, where vaccination teams go from the health facilities to places which are closer to communities that are poorly served by health facilities, and provide a day's worth of services on a regular basis in those harder-to-reach areas, making it easier for caregivers to access services without having to travel all the way to a health facility.

Elie Hassenfeld: I wonder if you can tell us a little bit more about some of the challenges in accessing services. So you said one constraint is that facilities are far from where people live. Do you have a sense of, just to help listeners better understand, how far? What would it look like to try and bring a child to a health facility? What could it look like for this roughly 50% of children who aren't receiving all the [00:14:00] vaccinations that they should in the DRC?

Natalie Crispin: Obviously there's a ton of variation in this. However, what we've heard as a rule of thumb is that those who are less than five kilometers, so about three miles from a health facility, are generally expected to bring their children to the health facility. Whereas, those who are further are targeted with these outreach or mobile services. That's not always true. Sometimes outreach goes to closer in areas.

But yeah, I imagine trying to carry a young child three miles each way to a health facility and have a chance, but not a guarantee of actually finding services there. And then there's some who are further out who may get outreach services sometimes, but not always, and their other option is to try to travel an even longer distance, to probably pay for transportation out of their own pocket. And the idea that they're going to do that eight [00:15:00] times in a year or a little more, is just a really heavy burden. So the fact that we get 50% coverage is maybe fairly remarkable, given some of those barriers.

Elie Hassenfeld: Yeah, I think another thing you and I have talked a little bit about is, I always feel like I appreciate the reminders I get from my pediatrician to go in and bring my children for shots. And it takes me, I don't know, five minutes to get there, and maybe 15 minutes while I'm there. When I make an appointment, the staff and the supplies are always in stock. And then just imagining the, I don't know, hour-plus journey each way, plus some chance that my child won't even be served if I undertake that journey. And it actually seems, not surprising, that the vaccination rates are low.

Yeah, something I think that is interesting about all of this is that in some ways, vaccinations have always seemed like one of the easier interventions. And I think they are maybe in the scheme of things. You know, with a water chlorination program, it's like you need to install this infrastructure, and people need to use it in an ongoing way. With a malaria [00:16:00] net, the net needs to stay in use, in good condition. People need to literally have it up every single night. And a vaccination is a one or a several time interaction with the health system that then confers these long-term benefits. But even with that, when you look at vaccinations as a whole, it's actually fairly challenging to achieve all the benefits that can come via this program.

Natalie Crispin: Yeah, it's right. I mean, vaccination does have its own challenges. So the benefit is that you do not have the same user compliance challenge. So vaccines, you just provide the injection or the oral vaccine, and there's no concern about whether or not the child is actually going to ultimately receive that thing after that interaction.

The challenge of vaccination, as we talked about, is the age-based schedule, the number of interactions that you need to deliver the full schedule. But also that you need a higher level of training for those who are delivering it. So providing injectable [00:17:00] vaccines is a skill. Knowing the schedule. Knowing the indications of when you wouldn't give a vaccine requires a level of training that goes beyond what those who are delivering SMC or vitamin A, for example, receive.

And you need a cold chain, so you need to keep most of the vaccines within a certain temperature range. And that means that vaccines aren't always stored at the lowest level of service delivery. They often have to be retrieved by the health facility staff on the day of vaccination in order to actually provide services. So there are some additional challenges, but they're a tool that has a lot to recommend them and, particularly for the vaccines that are a small number of doses and provide a very high level of protection, such as measles, is an incredible invention.

Elie Hassenfeld: I mean, it seems like with vaccines there are all these challenges, but you know what they are, and if you can overcome them, at the moment that the needle is put into the child's [00:18:00] arm, and you check that the cold chain maintained throughout, it's like you've had some positive effect. It’s just there, like it persists. And that is so rare, relative to so many of the other programs. And so if we, via the work that you're leading, we're able to support programs in overcoming these challenges, you can be very confident that you're having a material benefit on children health.

So this is the problem and some of the gaps. So then, how did you approach, how did the team approach trying to find ways to actually direct funds to organizations that would attack this problem?

Natalie Crispin: Yeah, so we've run two, so far, open calls for proposals, and we've received a lot of applications. From those, we've so far selected 12 programs, and this is with 12 different implementing organizations, six in DRC. That's by far the most in one country. And the idea here is to have an initial set of [00:19:00] conversations with the finalists who were chosen on the strength of their applications, and to understand more about how they would approach the problem of low coverage rates and pockets of population where children are particularly at risk, and how they're going to make it easier for those families to vaccinate their children.

We've had that series of conversations, agreed on an initial set of priorities that those programs are going to focus on, and then have really encouraged them to learn more as they go about what is the situation in a given location, what are the other partners, how can they best support the government in its strategy to improve vaccination coverage? And then iterate on the program design with the ultimate goal of making a meaningful change in coverage of these routine [00:20:00] vaccines.

And then alongside each of these programs, we're separately funding a series of coverage surveys. So, before the program begins and then annually after that, to see, is the program in fact improving coverage? If it is, we want to scale up the programs that are having the biggest change for the least cost and have this sort of series of bets where we pick the ones that overperform and scale those up and move away from the ones that underperform.

Elie Hassenfeld: So before we get into the nature of what these organizations are doing, can you just tell me a little bit about the organizations that came through the open call? How large are they in terms of... are they very large international nonprofits? Where are they based? And also, which of these were known to us before and we could have said, "Hey, if we're going to work on vaccination in DRC, we should probably support organization X," versus we learned about them via this open call.

Natalie Crispin: Yeah, so [00:21:00] there's a range. We had worked with some before. VillageReach was one of GiveWell's earliest grantees, and we had funded them, in fact, for a vaccination program, and we're now supporting them in DRC on a different kind of vaccination program. We had worked with CHAI before. They're a large international NGO. And then there's other organizations that we didn't know as well, maybe had heard of but hadn't worked directly with, particularly on vaccination, and have more local footprint, more integration with government. So a group like SANRU in DRC is one that actually our Malaria team had worked with some before, but we on Vaccination hadn't. And they operate, I believe, exclusively in DRC and have very strong relationships with the government and deep experience in health [00:22:00] systems there. Then there's some that are more in between, smaller organizations that have an international presence, and maybe more of a technological focus. So there's a group called Bluesquare that was new to us, and we learned much more about their vaccination work through the request for proposals process.

Elie Hassenfeld: When you're reviewing these proposals from groups that we've worked with or haven't worked with, what are the kinds of things that go into your thinking, your decision-making about which ones to support?

Natalie Crispin: A big piece of it for us is, do they seem like they either really understand the context that they are going to work in? Or understand it some and have a reasonable plan for how they're going to learn more and are displaying a level of curiosity and somewhat humbleness about what they can know from the get-go and what they are going to learn over time. So we [00:23:00] really want to see that they have plans for how they're going to collect data, both quantitative and qualitative, about what the problems are in a given place and potential solutions. And then also what actually happens when they start implementing. Do they, in fact, change service delivery and access for communities?

That's quite a bit of the process in terms of the individual organizations. As always, GiveWell is looking at the cost effectiveness of each program in a given place, but because we sort of started with that and had picked certain set of locations, that the filter for the organizations was more about their approach to addressing the problem of access in these locations.

Elie Hassenfeld: Yeah, that makes sense. So what is the core approach that you expect these organizations to take?

Natalie Crispin: So we've asked all the organizations to have a method for targeting, [00:24:00] and a method for reaching. So on targeting, it's how are you going to determine which communities and which children to focus your resources on? There are areas that are closer to facilities, facilities that are better functioning. And we want the organizations to not do a blanket approach, but to find the places that could best use additional interventions to give them access.

And then the other part is the reach part. And so, taking actions that will improve service delivery, both in health facilities and through outreach and mobile services. And so many of the, or really all of the partners, are doing something to increase the frequency and the quality of these services that are happening outside of health facilities.

There are many vaccination programs that really focus on helping governments design programs, these technical assistance programs. I think [00:25:00] those have the potential to be important, and also we wanted these programs to be more intensive and to go beyond the sort of planning and designing of the vaccination systems and actually provide additional resources to provide service delivery.

Elie Hassenfeld: And so, what are the kinds of things they'll do? What do you imagine happening when these organizations begin work, and what are the kinds of programs they could be choosing from, or the menu of actions they might take?

Natalie Crispin: So for example, a group called Acasus is working in one province in DRC, and they have designed, prior to our work with them, an application that health teams will use to log what their activities are. And so, part of this is to provide information to the health teams about what they need to do and to provide information to their supervisors about what's actually happening. Part of our grant to them will be to roll out more widely and support the health [00:26:00] teams to use it well and their supervisors to make decisions based on it. In addition to that, there's part of the grant that's trying to support those vaccination teams to actually go out and provide outreach services more often. And so they're paying for motorbikes, and fuel, and time for these health workers to be able to leave the health facilities more often and provide vaccination services.

Elie Hassenfeld: I'm trying to picture how the outreach services actually work. So, you know, I can picture what happens with, say, a big malaria net distribution because every household often receives multiple nets at this moment of distribution. How do these mobile outreach services go about solving the problem that you described at the beginning, which is that children need such a wide range of potential vaccines at a given time in their life. What actually happens to ensure that the young child who needs [00:27:00] vaccine X, actually shows up at location Y at the right time to receive vaccination X?

Natalie Crispin: Yeah. So I'll describe one version of this. They will generally bring a vaccination team, which will include a vaccinator, a data recorder, and sometimes a mobilizer to a community. And they'll often take motorbikes with a small cooler of vaccines where they have ice packs and a variety of vaccines so that they can provide what's needed to the children who attend.

So they'll arrive in a community. Either the mobilizer will be responsible for going to households, spreading the word. In many of the programs, there's also a community health worker piece where there's been communication between the community-based health worker who is letting people know that the outreach team is coming and [00:28:00] is feeding information back to the health facility about the number of children who are due for vaccines. So in those cases, people may have been warned ahead of time that these services would be provided.

The vaccination team will set up in a central location in a village, and caregivers will bring their children there. They'll have all of the vaccines, so they'll check the child's health card if it's available, or ask the caregiver what the child has already received if they no longer have their health card. Also, they may bring the records from the facility about which children have received what vaccines. And from that, they determine which vaccines the child is due for.

They'll provide the injectable and the oral vaccines. And then those teams, depending on the program, might come to that community, monthly would be probably the most intensive, or it might be every two or [00:29:00] three months. And particularly in the cases where they're not coming monthly, they're encouraging the caregivers to attend health facilities for the next vaccine. So they'll mark on the card, "This is when you're due for the next vaccines. Please see your local health facility for those."

Elie Hassenfeld: One way I was sort of summarizing in my head the approach we're taking here with these different organizations—I want to see if you agree with this—is that there are so many organizations that we're working with and that we're going to learn from, and then we hope that once we learn, we'll be able to scale up and deliver more resources to the approaches that are working best.

And then among those organizations we've supported, there's ones that are very large international nonprofits. There are ones that you described as being more quantitative and data-driven in their orientation. And then there are ones that are high context on the location in which it will be working best. And it's interesting to think about, I guess ex ante it's hard to predict whether the organization with the highest degree of local [00:30:00] context might outperform or underperform the big international NGO that has a different set of strengths and weaknesses that it brings to bear on this program. How do you think about that characterization? What do you think about that question?

Natalie Crispin: So we've definitely aimed to have diversity in this portfolio, because what we want to do is, as I was saying before, kind of like take a bunch of bets and see what comes out of that. Then we hope to really understand in a descriptive, qualitative way, what's going on with these programs, what's leading to better results, so that it's not just, “Well, the one organization that had a really close relationship with this government performed well, therefore, we're going to always focus on organizations that particularly outstanding on close relationships to governments.” I think that would be an over-extrapolation from a limited set of data.

So I do think we need to, in addition to having the diversity, try to [00:31:00] dig in through conversations with the grantees, with other stakeholders who are close to the situation, as well as through some additional data collection about not just did coverage go up, but what led to the changes in coverage? And then try to understand, what were the enabling factors for those actions that will help us gain a better understanding over time of what is driving success? So yeah, there's a lot we need to learn beyond, did coverage go up? And that's going to be a really big priority for us in the next couple years.

Elie Hassenfeld: This whole initiative, in many ways, revolves around us being in a position where we can learn and then do more in the future. So what are the things that you're most uncertain about? What do we most hope to learn? Maybe another way of asking this, as you mentioned that we'll be collecting both quantitative and qualitative data. What are the kinds of information we're going to get that you think is most likely to [00:32:00] differentially affect what we support next?

Natalie Crispin: Yes. So, I mean, I do think the change in coverage is very important, and the more that we can understand whether that change was caused by the program versus something else. So, having a comparison group; having information about what were the actions that led to the change, and were those driven by the groups that we funded or something else that was going on? That's going to be crucial. We're really focused on getting rigorous data on coverage and, whenever we can, having a comparison alongside.

We also are trying to learn more about what are the best methods for improving coverage. So, we've aimed for a diversity of approaches across these groups, within the box that we've kind of talked about, of a targeted access-improving program. But [00:33:00] organizations are doing different things. There's one organization that's looking at how much service delivery can happen through community health workers, and does that really change what we can accomplish? I mentioned Acasus’ program, which has an application that health workers use, and their supervisors can see what their activities are and make changes because of that. There's a group that's employing its own health team so that it has more control over where and how those teams deliver services.

All of the programs are collecting their own data about program performance, sometimes going all the way up to changes in coverage, but also steps along the way— what activities were delivered, what number of children were reached with those activities, and what number of vaccines were delivered to those children? We also plan and have started having conversations with others who are working in the same locations so that they can [00:34:00] give us more context—other funders, other implementers, government officials.

So while it's not an RCT comparing all of these different methods, I do think through following these programs, tracking the changes in coverage, filling in more information from conversations and from data collection about what happened along that theory of change between we gave money and there was a change in coverage, I am hopeful that we can piece together a picture of what are the best methods, the organizations that are having the most success, and how can we make this portfolio more effective over time. To help us with that, we're looking at potential external partnerships that can get into more of the details, design more robust data collection so that we're getting as much out of this series of investments as we can.

Elie Hassenfeld: So circling back to the start of this conversation, you said that this is one piece of what the Vaccinations team is doing, but this is not the entire picture of what you're working [00:35:00] on. Where does this fit into the overall work that you're doing, and what else is on your plate?

Natalie Crispin: Yeah, so we think that this portfolio—and I think it'll keep taking shape, it may not look exactly like a set of grants to NGOs in the future—but that this will probably be at least half of what we're doing. When you look across vaccine-preventable diseases, the set of vaccines that are delivered to young children have, by far, the most potential to decrease mortality. And so honing in on that set of vaccines, and increasing coverage, given the decent but still suboptimal levels of coverage, is a big priority for us.

Those vaccines themselves are funded through governments and through Gavi, the Vaccine Alliance, to pay for the vaccines themselves. And Gavi did [00:36:00] not raise all that it hoped to for its current strategic period, 2026 to 2030. So it does have some gaps. It's prioritizing the vaccines that are part of this routine immunization schedule for young children. There could be some gaps there, but that hasn't been the focus as much. There are some funding gaps for vaccine procurement, we think, and that is another potential area of priority. We've done the most so far in looking at malaria vaccines, and there's the potential to be significant gaps for the procurement, the buying of malaria vaccines, and so that's another big priority for us at the moment.

And then there's a set of additional vaccines that are on our list to go deeper on that are currently not fully funded and could potentially be highly cost-effective. So, that's another priority, is to do some deep research into those and understand the funding gaps.

Elie Hassenfeld: Yeah, and so even in this area of vaccines where there's an institution like Gavi that exists—it [00:37:00] raises a very large amount of money to support its work, it raised less in its most recent replenishment than it hoped—there are still these big funding gaps. First, potentially for procurement, like literally purchasing vaccines and getting them to countries, and then of course also all the work that we've talked about, finding ways to bring the vaccines to the people who need them, so they get them when they need them. To what extent are there others working in this space as a funder? What do you think about the existence of funding gaps in this area?

Natalie Crispin: Yeah, so Gavi is the largest funder in low- and middle-income countries of vaccination, excluding the governments themselves, which are huge funders of these systems. The Gates Foundation is another very large funder of vaccination work. They are a major supporter of Gavi. They also do a bunch of work to support health systems, complementing the work that Gavi does. And there are many other funders as well that are part of the space—WHO, UNICEF are both implementing [00:38:00] organizations and provide funding. But there do remain these gaps. A lot of the others in the space are more on the implementing side, and there really aren't major funders beyond Gavi and the Gates Foundation that can, at the current time, provide the level of resources that we think GiveWell and GiveWell's donors might be able to bring to this space.

Elie Hassenfeld: Well, thanks, Natalie. Anything else we didn't cover that you want to share?

Natalie Crispin: Yeah, maybe just to say that I think this is a really interesting space. We have a lot of areas still to cover. We're excited to be working with our partners in new ways and discovering organizations that can deliver the services that we've never gotten to work with before. So yeah, just really exciting times, and I'm just enjoying a lot, learning from others who have been doing this a lot longer.

Elie Hassenfeld: Well, thanks so much, Natalie. Really appreciate the time.
–

This is Elie again. I hope you really [00:39:00] enjoyed that conversation. I think it shows, in some ways, how GiveWell is very similar to what we've always been, and then in others, how we've evolved and are different today. We're similar in that we're asking lots of questions about how big problems are, and how we can cost-effectively address them. We're gathering data to try to make good decisions about where to grant funds and plan to obtain additional data after the fact, both quantitative and qualitative, so we can learn and continue to grant funds more cost-effectively in the future.

And then some of the ways in which it's different is that with a larger team and more experience, we're able to do the kinds of things that we couldn't imagine doing even a few years ago. We have a team that's able to put out a call and evaluate proposals from organizations that we use to determine where to move forward. That gives us a level of breadth in our vaccines portfolio alone that we couldn't have imagined a few years ago. We're also now covering vaccines at a much greater level of depth than we could have in the past, and we have this breadth of [00:40:00] coverage across so many different health areas at GiveWell. And then we also have the scope, the capability, the capacity to run experiments that can help us get better information about how to deliver improved health commodities in low-resource settings. We covered that specifically with respect to vaccines, but this is an approach that we're bringing to bear on all the work that we're doing today.

So I hope you found this helpful. I hope that you learned something about vaccines and the work we're trying to do, and better understand how GiveWell approaches the problem of trying to help people in need as much as we can. Thank you so much.