Welcome to the CAC Research podcast.
The series of conversations about projects taking place at the Centre for the Study of African Economies, University of Oxford.
My name is Stefan Darko and I'm the director here of the CRC and a professor of
economic policy at the Department of Economics and the Blavatnik School of Government.
Last week, the Economic and Social Research Council of the UK hosted their Celebrating Impact 2023 prize ceremony.
And I'm delighted to be joined today by the winner of the award for Outstanding Public Policy Impact.
Kate Walk. Kate is a member of the CSA.
She's an associate professor in Economics and Public Policy at the Blavatnik School of
Government and Leader of the Mind and Behaviour Research Group at the University of Oxford.
Kate, thank you for joining me. Thank you so much for having me.
It's great to be able to talk about research that seems to have actually had the
impact rather than aspirational research that hopes maybe to have some impact.
So it's great also for us to do this because, you know, as the centre, it's something we try to to achieve, to actually just do research,
not just as relevant for academic purposes, which is a virtuous purpose in itself,
but also to try to get further and see whether we can actually make a difference.
So I would be delighted to hear a little bit more from you about, you know,
what was what was the problem that your contribution tried to resolve here.
Tell us a little bit more about what was going on then what you are trying to solve.
So I'll take you back to the very early days of the COVID pandemic.
And so this this impacts of what happened in South Africa.
But the situation was very similar all across the developing world.
So in in April 2020, there were about one in five South Africans going to bed hungry.
And in many countries, you know, that lockdown, very hard work is very casual.
And so millions of workers were laid off. And if there was any government support, it has tended to be through food parcel distribution.
And so in South Africa, like in many places, the existing system of getting aid out was just completely overwhelmed.
There were issues with logistical management, there was theft.
And so you just saw these nightly images on the news of these long queues for food and they were all,
you know, running out and that that really dire situation was happening across the developing world.
We know these problems are quite predictable. So there's actually been a long standing evidence based in development economics over the last
20 years that tried to change what we what economists now think of as quite an ideological,
non evidence based approach to aid, which is that it has to go out as food policy makers worry a lot,
that if you just give recipients cash, they'll spend it frivolously.
And so they either give us food or if welfare is given as cash like to jobseekers,
it often has these quite strict conditions attached like you have to you have to be searching for a job.
And so in this policy situation, the existing research base that we had, which are very strong, is quite a radical new form of welfare.
It's the idea that you can just give cash directly to poor households, including during emergencies and pre-COVID,
you and I, as well as big team of other researchers Rob GARLICK, Marine Mahmood Richard or Maya Johan House offer.
We ran this big randomised trial in Kenya with GiveDirectly,
this NGO that has really pioneered this approach to test what happens when you just give cash directly to poor households.
And like a lot of other studies, we find that recipient households really use the money well quote unquote.
You know, they spending it on food that improves children's nutrition and development.
They also using cash to buy assets or businesses or to search for work.
They often working, in fact, more rather than less.
And they're certainly not wasting the money.
They, you know, really investing it in things that can help them to improve their economic position in the future.
And so what this research was trying to do was apply that evidence based, which we had pre-COVID,
to the crisis of aid distribution that was happening during the COVID pandemic and say, look, actually, we know a shift that needs to happen.
It's a really big shift. But we we need to stop trying to think we can get food parcels out and that that's going to solve this emergency.
We need to do something that the policymakers may not approve of.
But if we can give aid as cash, we'll be able to get it out much faster and to many more people.
And that's that's really interesting because if we think about it, you know,
the evidence base and as you also allude to around, you know, that cash can be really effective.
It's been there now for a while. And it's it's an interesting.
That every time, again, when you try to to do some of these things and you tell policymakers, well, we have the evidence, you can just do cash.
I know from experience as well, a lot of people will have this prejudices.
And and it's not just, you know, governments. It's middle classes.
There's all kinds of people have these prejudices that actually giving cash will have these not beneficial impacts.
So so it's one thing to supply the evidence, but clearly needs other work as well.
So how did you go about convincing them? And I think that's the more almost the more important part, because we can keep on writing these papers,
We can write a little block, we can say this, we have the evidence that we've seen so many countries that it's not been picked up.
So tell us a little bit more how they came about and and how did you go about this?
So I think the most important thing is that this could happen because there was this huge evidence.
Researchers tend to get excited about the new study and their working paper.
It's not even published yet.
And then they want to go and say to the government, Oh, you should do this new thing at the frontier that I've come up with.
And I think the key thing here was that we were able to say, you know, there's been systematic reviews.
They know 160 studies all around the world in many different contexts that have similar findings.
You know, we've done it recently in a context like Kenya, which is very similar to this country, but actually the evidence base was big.
I think for me, that was that was important. In fact, possibly governments shouldn't be scaling one study.
They should be scaling things where we have meta analysis, where we know that the evidence base is deep and it's uncontroversial what to do.
So I think that was the first thing. It was really important when talking to government that we weren't just shopping for solution,
we were doing evidence reviews across the broad range of the evidence across multiple contexts, including contexts that were similar to South Africa.
To draw the conclusion. So I think that was the first thing was, you know, the evidence base is actually going to be quite deep.
The second thing was, you know, changing format from the what a research paper was into what was actually going to be useful in the policy process.
So the way that the impact happened,
we work collaboratively with the presidency and the Social Security Agency and the Department for Social Development in South Africa.
And there were a group of civil servants who knew of this evidence base,
and they formed a working group on poverty that wanted to see whether the existing grant system in the country could be used to,
you know, put in place these solutions. But the really important thing was workshopping the policy questions that they needed answers to,
thinking about the formats that they needed the answers in.
So it wasn't a research paper, it wasn't a policy brief. It was we're going to have to go through Cabinet resolutions.
So we need a sort of short summary of of what's being proposed.
And then we need the underpinning research work written in a really kind of clear, accessible format.
And I think learning to do that writing was completely different from how you would write a research paper.
You know, it's really focusing on what the evidence is across the range of papers rather than just one paper.
So, you know, you're writing statements like the 20 studies in this field and six of them find this and, you know, 14 of them find that.
So, you know, our best guess is that this is the this is what one should learn from that evidence.
And I think that's really something economists don't do a lot in health.
We do a lot of major studies, and we've actually worked a bit with the Mind and behaviour group on doing some some meta analysis studies.
I think that was actually the most important skill that I was able to bring to this was sort of looking across the evidence.
So I think that was the second thing.
And then the third thing, which is something the CSA has been, you know, really worked at very hard, is having these deep,
long collaborations with economists in developing countries so that you here we were working with the university in Cape Town.
That was actually where I did my undergrad.
So working with a team led by Ingrid Woolard, Mary Labour, and then we also had Maya Goldman to to Coachella a Jessica Nicola and Brandi Kraft.
So it's a really deep team of both senior and more junior researchers who had huge government reputation.
So they'd done consultation for the Treasury and the presidency before Mary was on the big panel that looked at social welfare even pre-pandemic.
And so, you know, it was working with that team that we were able to achieve policy impact because we had local credibility.
So I think those kind of three ingredients really were what brought together having the impact.
But it's very different from what the current model is,
particularly in the world of you do one child policy brief and then government is going to do the policy.
This wasn't what it looked like at all, and I think that was a really important learning for us.
So kind of push you a little bit on that terms of the way these debates would
have gone and the way people were thinking about what is that evidence base,
you know? You know, I don't know if any of the studies you could review referred specifically to South Africa,
but often you get this experience that, yeah, maybe that may have worked in, I don't know, in in Ethiopia or in India.
We want to work here. You know, we are different.
You know, there is no external validity to this or surely the cost involved here would be so much higher.
What are these numbers involved? So was there any attempts to do some modelling work this some some adjustment work?
How how was that handled? Was that something the Treasury people were doing?
Always other people doing. Tell me a bit more about that. And then how do you convince bureaucrats?
How do you convince civil servants? I've been one, you know, it's not easy to convince them.
So. So how do you think? Tell us a little bit more how that all went.
I think first focusing on does the international evidence apply will not mean in this particular instance.
Actually,
the government was hugely eager to get that comparative evidence base because South Africa had this long history of doing cash grants for pensions.
And there's a big child support grant that was put in place that's been enormously successful.
So, you know, we didn't know about running cash grants and that that program is actually kind of world
leading in terms of the number of beneficiaries who had managed manages to reach successfully.
It's very well targeted, it's extremely pro-poor, and there's not a lot of leakage in the program.
It's works on biometrics. So there was that success case.
And I think, you know, had we been starting with the social welfare system from scratch, that probably would have would have been difficult.
But they were incredibly eager to learn from other countries.
So one of the most focussed on pieces in the policy brief we had set during the early days of the pandemic,
we actually got a Venezuelan student who could read on what the Latin American countries were doing,
you know, when it was in Spanish and Portuguese, you know, we couldn't even get it in English.
But we had this comparative table of what are the different countries doing at the World Bank?
So Hugo Giardini and that sort of social protection group, they were talking a lot about the evolving response to the crisis.
And policymakers really wanted that because I think one of the things they were worried about was, you know,
are investors going to think we're being completely profligate, just giving out money to our populations?
And the fact that there was this broad movement towards increasing social protection during the pandemic in
other countries was really important to reassure them that perhaps they wouldn't be that response in this case.
So I think people were really eager to get the international experience.
Actually, one of the things Marianne and Ingrid and team had worked on, you know,
even in previous iterations, the work that they'd done was a deep mission,
kind of talking to the people who did Bolsa Familia in Brazil and learning a bit more about how they did the targeting,
because the approach that South Africa ended up using is the most similar to Brazil compared to other countries.
So I think there was a real eagerness to learn from other countries.
Yes. And not to sense that this wasn't relevant. And we have to have the evidence for our context.
You know, policymakers are quite used to making the decisions based on not perfect information.
I think sometimes we in especially in the world, we think, oh, we have to have done the audacity of this trial in this context at this time.
And they were less worried about that. You sometimes having to make jumps, you know, and you don't have the perfect evidence.
The other thing that you asked about cost that was super important and, you know,
we did some cost calculations in our initial Kenya trial, but I didn't know anything about studying costs, which is a great shame.
I think, you know, what the U.S. team brought. You know, they're not primarily ICT researchers.
They weren't studying cash transfers.
They had already built this model of the South African economy that one could use to study the poverty incidence of social transfers.
And you know how if you also wanted to put in place tax increases, how that would affect levels of poverty.
And so that wasn't a modelling technique I was familiar with as someone who primarily does field experiments,
but that was the core of the work was being able to say this is how many poor people they are, you know, people below the poverty line.
If you give this amount of money, this is how it's going to change what people's consumption levels are, whether they are living in poverty.
And then if you do different versions of the grant, this is how many people are going to be eligible and what the poverty impact is going to be.
And so that model was the core of the work that we did, and it was a real learning to me as an ICC researcher.
You know, the benefits is only the first step.
And then you have to be able to say to the government, who is this going to reach and how much is it going to cost?
And those are not tools. We've even, you know, we just cost what's in the act.
But that's it's not even the right question.
So I think that really blew open for me what the discipline needs to be doing if you're really going to get national governments to scale.
And I think it is collaborating much more with researchers in the sort of fiscal and public space to say,
you know, how is this policy going to play out at an at a national scale?
That's really interesting. Clearly, you know, it's one thing for a government to take up a policy and to do it.
Let me gently suggest you wouldn't get an impact prize if there is no evidence that actually it
had impact and that actually there is some evidence for South Africa of of the impact it have.
You know, that there's there's one thing that I don't think I've ever told you really is that
it was a peer review or actually over review on on the impact on poverty across the.
From the COVID pandemic run by the World Bank.
And it was really interesting because they just crunched the numbers from all of the world, whatever they can come in.
And they kept on being worried because South Africa was an outlier. They said there must be something wrong with the numbers here and so on.
And actually only in that process we could see and maybe there's one or two other countries that actually really
have managed to to have a substantial poverty impact during the pandemic in ways that they were quite surprised by.
And so so they ended up digging deeper. And and it is the case that seeing from the World Bank clearly from their comparative evidence that South
Africa ended up being judged as one of the countries that better handled the poverty impacts than than many,
if not most countries across the world. Now, I presume you will have other evidence.
I'm sure the government has been trying to collect it.
Tell us a little bit more what is the impact on beneficiaries and what kind of things have we learned since the basic policy shift?
So we had the situation where they were giving I think they were giving out about 1.2 million food parcels a week,
but they were about 10 million people who were below the breadline.
So below the food poverty line, they didn't have enough income that they could actually get enough to eat.
You know, the need was about a 10th of what was going to be necessary to avoid widespread hunger
and what the government did in that situation instead of sticking with the food parcels.
So the food parcels were actually constitutionally set up approach to social welfare.
So, you know, like many developing countries, they didn't have a basic grant.
If you were an adult, you were able bodied, so you didn't have a disability, but you were unemployed.
And so the only thing that was there was these food parcels.
If you were really in very dire need and you could go to your municipality and apply for them.
So the shift that you know, with the city researchers and our team at Oxford,
the shift that we achieved was instead of delivering aid through the food parcel system, the government did two things.
They first increased the amount that was paid to people who are already getting a cash grant.
So pensioners, women largely who were receiving child grants on behalf of their children and people who had a disability payment.
So they temporarily increased the amount of the cash grants.
We know already from the evidence that people share that money with their families.
So the idea was, you know, already through the grant system, just by increasing the amount that flows out, you'll be able to reach more people.
And then the second thing, which was the really, you know, remarkable policy shift,
was to do this new monthly cash grant for 10 million able bodied unemployed people.
They had previously not had any welfare payments, but the government would put in place an entire new grant in a six week period.
And the technology was amazing.
People mainly signed up for it for WhatsApp via WhatsApp, but they could also sign up online and so they signed up for the grant.
Then you had checked, you know, this was the Brazilian approach.
They checked against a set of data that, you know, a few things that they could check in Edmond data that you weren't very wealthy,
you know, you didn't have a vehicle, you weren't getting unemployment insurance, you weren't on the government payroll, for example.
And then later we actually built in that they looked at people's banking data to see their income,
and then that that new technology meant that they were able to get eventually about 10 million people onto a new monthly cash grant.
And so all in all, that meant that we went to reaching from, you know,
1.2 million people who were getting the food parcels to 28 million people who were getting some sort of payment from the cash grant.
So that was a huge shift in social welfare spending.
In total, governments now spent about £7 billion, so that's about 3% of annual GDP.
It's about the same scale as the as the UK furlough scheme through that kind of increase in in cash grants.
So that's and that the research has shown that that's really been targeted very well at the country's poorest households.
So the sort of set of testing means testing that they did has actually managed to have a really sort of poverty targeted grant.
And so now they aren't they're no longer doing the increased child and child grants and pension payments,
but that grant for the unemployed has stayed in place.
And so we've we've modelled that with the team and we estimate that that's currently keeping about 2 million people out of severe poverty every month.
So it's been a really big new kind of policy shift.
It's the first grant for the unemployed in Africa.
So it's it's been a really big extension of the social welfare system and it's really having quite.
Remarkable effects on poverty. That's that's really fascinating.
Maybe kids. As a final question. We're all researchers.
We all try to have impact at times.
What have you learned about trying to get impact from research from this experience?
I think the the most important thing in this was was empathy.
It was about stepping out of your academic world and trying to put yourself in the shoes of the civil servants who were trying to solve the problem.
And so we worked really closely with this group. They were just incredibly committed to deliver that kind of impact.
They were just working round the clock for years.
And so I think so to really say, if I was sitting facing this problem, what are the technical inputs that I would need?
And you have to take yourself as a researcher out of the process.
It's not about whether it's your particular thing that's got your name on it.
You know, it's not about whether it's your particular solution. It's about what, you know, what's actually going to be the best in the situation.
And I think drawing on the really broad range of evidence, looking at what, you know, other people in other countries have done,
and then really thinking about what others, the civil servants worried about, Why are they worried about that?
You know, what are the barriers to them, you know, taking this this policy to scale?
Because I think that's thinking partly about the technical challenges.
And, you know, they were really deep in the weeds of the numbers and the model,
but trying to make the model so that it answered questions that they had.
But then also thinking about the politics, like how are you going to get this grant through?
And so I think this for us, there's this huge worry that the cash grant builds dependency and that's a you know,
that that has been there in wealthy states for hundreds of years.
But actually to have this evidence base, we don't talk about it emotionally, but it's to say, you know,
giving people cash builds resilience, it builds autonomy, it builds people's ability to make decisions.
And, you know, they are the owners of the economic lives. And, you know, you're giving them the most flexible tool that you can.
And that narrative of empowerment was hugely politically important.
And so once we realised, you know, wasn't a lot of the evidence is saying just your food, calorie consumption improves.
But the thing that actually gripped people was people can look for work, they can buy assets and that they weren't reviews on that so much.
And we really focussed on that evidence base and I think that was actually what swung it.
And that's also been what's kept the grant in place as part of the economic recovery is,
is saying this is a tool to help, you know, where the state is struggling to get that sort of economic growth going.
You actually putting this in the hands of the citizens and those are very effective economic agents.
So I think that change in narrative was actually the most the most important thing, and that came from China.
Understand the political reality that people were facing. Yeah.
So I think empathy, empathy with your policymakers is my my catch phrase.
Very good. Well, it will come. Thank you very much for talking to us.
And also, again, congratulations for winning the award of outstanding public policy Impact of the Economic and Social Research Council of the UK.
Thank you very much for talking us.