I would like to start by having a little bit of a discussion of some of the bigger issues that come out from this and then turn it to
the floor because I think it's really important to give you all the opportunity to interrogate the issues that matter most to you.
Rich, what's the extent to which I think the research we're doing and the challenges we face,
it impacts on how we how we demonstrate that resonates with your own experience.
And I think it's helpful to start by just, I think, highlighting some of the key points that came out,
one of which was the organic nature of change and of impact in policy.
The fact that things don't policies don't happen in a linear fashion at all.
And the sort of almost serendipitous nature of, you know, picking up and running with something that you've been commissioned to do,
say, by governments and the way in which that then is enabling you to be heard in other fora and elsewhere.
It gives you the authority and the credibility that you need to be able to show that you can actually contribute to to to policy.
And I think it's really important also to emphasise, perhaps it was a little who bought it more out than the other to two presentations for Ethiopia.
The extent to which when we started the world was not really geared up in many countries
to listen to research evidence at all and to use it in planning and policy in practise.
And I think that's now more established than it was. But let's not assume that it's the only source of influence on on policy practise,
and I think we're often competing with all kinds of other force forces that also influence policy and practise.
And of course, there's lots of spurious evidence out there as well that we're competing with.
It's not like we're the only player on the block, and I think you're probably all facing that sort of experience on the field, many changes.
And I think it was really interesting that I bet many of you face working with theories of change and logical frameworks and so on.
All these tools that are now the sort of magic ingredient, theoretically of impact and so on.
They didn't exist when young lives were set up. It had a log frame, but it didn't have a theory of change.
We now have a log frame and a theory of change, and I've just been reviewing them and realised they don't really speak well to each other.
So it's incredibly easy to find yourself developing these, even in your own agenda.
These things almost in parallel. So I would like to turn back to the three of you and just ask you if you were to be starting young lives.
Now, with the funding starting from now and everything else, how would you would you set things up differently?
How would you do it in order to achieve the impact that you want to achieve?
Bearing in mind that governance is also we've argued that governance is a key issue to that partnerships, relationships and everything else.
How would you do it differently or would you do it differently? What did you learn?
What's the big one income you have? Nice.
Well, let me start with one of the things I I have been thinking about is Young Lives was set up from the beginning as a study of childhood poverty,
but a multidimensional so we had professionals from different disciplines trying to work together and integrate their work.
But that meant also that some of the surveys we had lots of topics.
And so I have always wondered, I'm not sure what the answer is, perhaps somebody in the audience can speak to this if.
What would have happened if we had decided on a few topics, more select the topics instead of the range of topics that we have had,
we have also changed over the years and that's something I keep thinking about some issue.
I don't know. So I think I wouldn't change the number of topics.
I would keep those because that's what lends itself that, you know,
you're looking at different domains, not a nutrition focussed study or an education focussed study.
And you can look at all those other determinants of, you know, say, child marriage,
which cuts across education and federal education, menarche and a whole host of variables that you get from our dataset.
But I think what I would do is if we had the money, of course, would have wanted another cohort in another state,
you know, or maybe two states which would really, you know, speak their northern state and the southern state.
And maybe, you know, in the northeast and then be able to look at how children grow up in poverty and children,
I think renamed the study children and youth in poverty because we are,
you know, so looking at young people, you know, because we had nostrand, there were 22 not children anymore, actually in round five.
So yeah, that's that's that's something that I would definitely.
I think one other thing maybe I remember going in, you know, in round four to, I think,
meeting somebody in the Ministry of Health and looking at really trying to bring
the dialogue between teenage pregnancy and child marriage together because they,
you know, child marriage is seen. As Ministry of Health says, this is not our problem, right?
Teenage pregnancy is but not child marriage and Ministry of Women and child and Child Marriage marriages our issue, but not teenage pregnancy.
Go to the Ministry of Health for that. So we to talking to them and they said.
Here in round four of. Did you take permission from us before you started the study in 2002?
So, you know, I think what's really important is, you know,
how do you make sure and I guess it would be even more difficult in today's political climate,
but I think just getting a buy-in and ownership from the government and a bigger stake without taking money from them,
of course, so that you remain completely independent and objective.
But I think right from the beginning, getting us a buy in from the government would be really important, even in the way we would roll out the study.
Yes, I would echo what both Santiago Santiago and Ronald Reagan were saying, you know that from the start,
having a much more consultative process would have been would be very would be very useful so
that the objectives and talking it through and trying to work closely with with what's going on.
But a lot of it's very unpredictable.
And I think the holistic aspect of and looking across different sectors in the different domains is really very crucial.
And because it's a study of childhood and growing up,
it's clear that in the early years nutrition is going to be important in education is going to become much more important
as the children grow and later on the questions of marriage and setting up households and moving on now to work,
it's a logical progression.
And so I would have I think it was really important that we kept that and we can't you can't really also predict what's going to come up,
come up as a as a definite issue.
So I remember when I first joined in 2009 and and Joe said to me, when you write a paper with me on FGM C and I looked at her as if she was mad.
I mean,
the children were only nine and this was something associated much more with marriage and it was several years down the line and and it didn't.
I had no idea that the girls summit would explode onto the scene, that this would become a commitment to the Egyptian government.
And so I think there is a danger of of trying to define.
A primary and beforehand, what you're going to look at, and that's one of the beauties of a much more open ended,
less narrow and more holistic approach, such as as young lives had.
I think as with renewal in our case in Ethiopia, not including the lowland agro pastoralists emerging regions.
In my view, was a mistake. There were it would have been risks of attrition.
But I think that that that's one of the things I would have changed.
I'm very interested that nobody's really mentioning almost the elephant in the room because for me,
one of the biggest challenges to having impact on policy in practise has always been the
volatility of the political environments in which we're working and the extent to which I mean,
I've just so many times we have worked so hard on a particular ministry, on a particular official.
And we've really built trust with them. We've built our credibility with them,
and we've just got to the point where we think that they're going to make a really great decision that's based and informed by our research.
They move on. They're shifted to a different ministry or the ministry policy changes or and it's likewise with the donors.
So you're building relationships with donors as well.
You're building multiple relationships, both with donors, with with exclusive tenders nationally and so on.
And the donors sometimes residing in in the states or the UK or wherever they are.
They they may also say, Well, no, this is no longer our priority.
So you're in this incredibly volatile environment where your constant, your basic research questions, your research issues are constant.
That's the whole point about longitudinal study. We talked about the ability to be to be able to adapt and adjust and so on.
But it's also the case that you've need to remain faithful to your original design.
So how do you deal?
I think with that kind of volatility, I think is another really big issue because we never know what's around the corner in terms of our politics.
And that's the politics actually in the UK, as much as it is in the study countries, because we've seen a lot of political [INAUDIBLE].
I mean, just to tell you all that young lives was about to go into its most expensive year ever in terms of data gathering.
Just at the moment that we had a 2016 referendum on Brexit, that you might all think that's completely irrelevant to a study.
Like ours, it crushed the pound, the value of the pound just the moment when we were about to shift all the funds out to the study country partners.
For us, it meant hundreds of thousands of pounds knocked off our budget instantly.
So that's the kind of, I think, just to say a couple of things.
Maybe if you cared about how you dealt with those unanticipated changes, some of the threatening and risky some of them also offer up opportunities.
And then I'd like to open the floor and and ask people to ask questions.
Well, one of the things I hit,
I think Trent was suggesting this is that you're not seen as an ally of any specific party or person or political institution in any way.
You're saying you sort of gave your credentials as a research institution that is kind
of say what we find in in our topic because we know these changes are going to occur.
And fortunately enough, we have been in a position whereby people that replace those who have left are known by
somebody within the institution and so we can re-establish links and pick up from there.
And that's that's a work over the years. But. So I think remaining apolitical and being seen as apolitical and even though I think that's very hard,
but and I think keeping your interface more to the bureaucrats rather than the ministers.
That's that's how we we we deal with it before you. Your bureaucrats don't change.
I mean, they do change, but one bureaucrat replaces another bureaucrat. But if you're aligned to ministers who are political from political parties,
that's when I think it becomes very difficult and the next party in the next election.
You may not be even allowed entry into the policy forums on the tables where you want to be.
So I think that's one thing. But yes, it's it's extremely challenging and particularly in recent times, for example, when you know you are told that,
OK, so we won't come for dissemination of research reports, but will come for dissemination.
So just make it into a fact sheet. Yeah.
So yeah, I mean, I think the different times, different challenges, and you've just got to find ways of saying, OK,
but you know, as long as I think to my mind, I think our biggest like Al-Ula referred to it when he was speaking.
I think the fact that after so many years, I think we've been around for very long.
You know, we are told by people, OK, you do on this research, I think that that itself just stands by us.
So they know we can't be pressured into saying things. I think nobody even tries, and I think that's a really, really good place to be in.
So I think the that the turnover issue, both in terms of government staff, donor communities,
researchers is a real challenge and that's one of the advantages of being there for a while
that you unknown and then new people coming in want to learn from what what you're doing.
But it is a challenge. So for instance, the Ministry of Women and Children had youth in it, and a separate ministry of Youth was created in Ethiopia.
So we were engaging with the Ministry of Youth. And before we knew it, the Ministry of Youth was re folded back into the Ministry of Women,
and that creates tremendous changes or the main research organisation that did the survey in Ethiopia this year,
Investment Research Institute got merged with another institute with a very different way of operating.
So. And that transition, and again, it's a credit to Oxford that they saw the importance of retaining the relationships with
individuals and institutions and working through very difficult contexts of change,
political change and so forth. And I think if there was one element that I would like to see,
I did more and I know Joe did struggle to try and push for that a political economy approach
to the way that research is done right from the way that the research is conceived,
designed, funded. Understood the way that we actually present our messages.
The whole political economy of that right down to the way in which the children and the parents and the communities,
the way the communities experience this.
Understand this. React to it. You know,
we we have this rather an a rather sort of assumption that our research is valued
a value neutral and that we're doing this in a net neutral scientific way.
But of course, we are going talking to young people. Their views are changing as we talk to them, are in dialogue with them.
And I would I would like to see a much more reflexive approach about the political economy of knowledge generation,
knowledge sharing, knowledge dissemination, knowledge contestation.
So I think that that would be something that there is avenues to to really think about the way in which research on poverty,
research on childhood research on communities, how this is contested and understood and represented.
And so that's really there's a lot more to be done that.
So I think that's a very nice way to end this, but also the exercise,
one of the I think one of the things that we try to do and I can't say that we always do
it successfully is we do make a clear distinction between conceptual or knowledge impact,
knowledge change and the more instrumental impact.
And I think probably the kind of research we do is observational research is much better.
It's conceptual than it is at the instrumental. So Lula mentioned of, you know, randomised controlled trials, they are much more instrumental.
They can go right down to chasing, you know, what an impact of a specific intervention has been,
whereas we're much more likely to shift the way people think. And I'm often very excited when I see.
I mean, the best way of saying that war is when people claim your ideas for their own.
And that's great, because then you know that you've you've got so much into their heads that actually,
you know, they've they basically don't even know that it came from you in the first place.
So those conceptual shifts can also then begin to shift the way policymakers do their business, the way they do that, that planning and so on.
Thanks so much, it's so great to be here and hear about all your experiences across different countries.
So my name is Catherine Gresham. I'm a researcher on a large, difford funded multi-country research programme as well.
And something I've been increasingly thinking about is ethics in development research,
and I would be really interested to hear how young lives approach the idea of ethics because ethics,
understanding of ethics and research differs across different countries. And did you?
There's different ways of thinking about it in terms of very formal ethical approval procedures that you need to go through,
but then also broad ideas of ethics around impact, feeding back to research participants and a reflexive approach to research.
If you would mind commenting on how you dealt with that in your project request.
Thank you. My name is Kathy Silver and I'm from the Education Department.
I've been one of the leaders of a very large scale longitudinal study just in one country on education.
It 3000 children. It lasted 19 years.
Joe's final elephant in the room really touched my heart because we reported and worked very well with the Labour government for many, many years.
Suddenly, we had a very conservative government and we had to really retune the team and
we had to reallocate the budget because we knew that with the new government,
certain areas were not going to be accepted and we felt we could somehow slip into other areas.
That meant that certain people who were promised things and were expecting things to happen, those things couldn't happen.
It also meant that you moved to New Delhi.
I mean, that would be a sudden expense that wasn't planned. A cow on your teams?
Do you really live together as a coherent team with respect for one another and yet make these changes?
Because although I think Santiago began saying that basically things are getting better for the children, there's catch up.
We have a positive story and that's the one side of impact.
It's kind of the research side, but the policy side with governments is not a story getting better.
It's a story that gets better, gets worse, gets better, gets horrendous. And how as a team do you deal with with having to make shifts?
So you really stole my question. It was something that was a burning issue.
I was I had one. Well, the city of you were talking, so I'm not going to repeat the question.
So just to say that I have I haven't done longitudinal studies.
I have done very short term studies and I have experienced that volatility that you mentioned is really destabilising.
But I wonder if the answer to that because ultimately you don't engage with government departments, you don't engage with policy makers,
you engage with specific individuals and you build relationships of trust and at some point, those individual relationships.
But that individual capacity building that you building that particular government department has to become institutionalised.
And I think it was you, you mentioned about culture, shift culture, cultural changes within government departments.
And I wonder if you have reflected in how that process has taken place,
if indeed it has from, you know, those individual relationships with particular chunk,
if you like to the more institutionalised approach and embedding in this case, young lives in government.
So my name is Mysskin, not an epidemiologist at the Georgia Institute of Global Health.
My question is regarding the study sites.
So I was wondering if the study sites where well told over, especially with regards to it.
Yeah, I was looking to take to the geography where the data was collected.
And I so like, for example, there is one region where there are about like four study sites, whereas another region, there is almost no study site,
which I think basically is an important study region, which is a Somali region, for example, that is basically affected by drought for many years.
And so it's just a little bit, yeah, unexpected to see no data was collected from.
Such kind of thanks. Such a fix, it seems on first all the time, anyway, we'll switch next week.
OK, OK, I promise about ethics.
Yes, we go through the formal processes, of course, how we have been through the Ethics Committee in Oxford and in study countries.
But we had to go beyond that.
And there's an interesting exploration of this in a tough in our publication by Virginia Morrow in our in our website, if you want to look at it.
But things we did. We felt that if we gathered so much information, we had to give them back some of the information.
So we give back information to local authorities, for example, but also sometimes to family.
So for instance, we had information about growth and weight, and field workers were trained in Peru, at least to quickly assess how they were doing.
And we had a set of recommendations if they were undernourished or or obese.
Of course, if they were too thin or too fat, then it would become something else and we would deal with specific treatment.
Or in some cases, we have of four for the mouth when they were growing older.
They really appreciate consulting we have on public services that they can access to on scholarships, on health and on legal advice and so forth.
We sometimes we hear that this may be an intervention, but you know, we thought we had to give them back something.
And of course, there are the individual ethical cases where we meet and decide,
how is it that we're going to go about somebody who says thinking about committing suicide or things like that?
Because even though it's not part of the questions, field workers that go back and back and spend,
if you are with them, you gain the trust of the people that after some point and say, I'm going to tell you something.
This child is not really my child or or my husband is violent with me or whatever.
And so don't tell anybody. But then we start discussions.
Can we not tell anybody with if? And so we go case by case.
And this thing's very challenging the question posed by Dr. Silber.
In fact, my first reaction is I would love to hear what you did in your longitudinal study because you have to make changes.
All, like general studies, keep changing and they will only be good if they keep changing.
But it means you have to disappoint people. Is it going to be made even more money?
So we struggle. Yeah, but I think if you have a contract signed in stone, I think it is that short term contract so you can make changes.
So. Yeah, difficult. Yeah, it is difficult, of course, I'm going to say only a couple of things.
We realise that I am in Peru. Even the bureaucrats change.
We have no public career, so it's very easy for somebody to be in an office and say everybody here is fired
and we're going to get other people from my party or whatever to be replaced.
Sometimes, no. But it's true that the higher posts are more political than than the less responsibility.
And oftentimes we engage not necessarily with a minister for some events.
Now, if you want publicity and the press will come, then you get the minister because you're going to have filming on TV and newspaper and so forth.
But for other more technical issues, perhaps. So you have to choose and thinking about that.
One potential answer to your question, I guess,
is to try to reach several audiences because if you only think about policy makers, then the things might be frustrating.
So that's why I was mentioning that we bet a lot on the next generation of professionals
to use these materials and to learn that they can be critical users of research.
I try to describe them, not necessarily researchers, but they can read a paper.
They can understand what it means and they can think. How does it challenge current policies or theories or whatever I do?
What could we think makes or they come up with suggestions for that, for the study, you know, so I don't know.
I'll keep thinking about it. But these are things like you said that we start with and I just sorry, we don't need to answer every question.
No, no, no. I'm done. So I want to pick up on the reason I'm interrupting in ways because I just want to pick up on this point.
One of the things we've tried to do recognising that some of our colleagues
in the study countries are working very challenging political environments.
We've actually sometimes tried to overcome this problem by having the publication of a message come out from somebody else.
So, for example, some of the messages around ethnicity and ethnic differences and so on are very difficult in Ethiopia, for instance,
because there's great sensitivity about really increasing the risk of of tension, ethnic interethnic tensions and so on.
So sometimes it's easier for somebody in Oxford to publish a paper which really raises this as a serious policy issue than it is,
say, for the Ethiopian team. So that's one of the ways in which I think we've sometimes we don't necessarily get it right every time.
But I think the advantage of being in a team which includes people from many different places and different institutions is
that it can come at it from different angles and have different people disseminating those ideas so that it reduces the risk.
I think what we're always trying to do is stay with a place at the table and you don't want to jeopardise that,
a very valuable place you have at the table by, you know, giving something which is contrary to current orthodoxy.
But that's really hard because you're dealing with governments that are often extremely unjust and impartial in their thinking.
So I just wanted to put that in. I don't know if it's a do you want to respond on that same issue, the two of you or do you want to move on to one?
I think I think there was a question on the embedding of, you know, how we look at institutional shifts.
I think I'd move on to that. I I don't know whether we can ever claim that we had a cultural shift to the ministry.
I don't think we could ever do that and I wouldn't even go near that.
But I think what you can do is, for example, just this is an example.
There's a new national policy that three draughts have been out now of education, right each.
So what could we do? We could really build a lobby so hard because it's not just us to listen to.
It has to be many more voices around the table.
And I think one way of working and bringing about those cultural shifts is having a much larger lobby asking for the same thing.
So because we now wanted secondary education to become compulsory because we realised that elementary education in itself is not enough.
So how do you create that lobby, including wanting to keep that focus on early childhood education?
But the the current politics is such that we know women and child doesn't want to give up the piece, they want to keep it there.
So how do I think? How do you traverse all of these? You know, there are budgetary implications for ministry.
It's a lot to do with all that. It's about egos. It's about it's not easy.
I mean, I think it's not an easy answer to say, but yes, I think what helps is so if you have multilaterals also saying.
Asking for the same thing all governments will listen to because they do get some funding,
even though the government may say it's very small amounts coming from the bank or elsewhere, but nevertheless they have a seat on the table.
So it's working with those influences so that you build that voice and together you try and build that change.
I don't think we can ever claim to have made the change ourselves.
Not at all. All very.
All the questions very challenging, and I think that we haven't got perfect answers.
But let me have a go at each one of them, so I'm the only ethics.
I think the climate of ethical approval has changed quite dramatically.
So now, you know, when we started off, the ethical approval was largely through through Oxford.
Now it's at least to me. Koepka in-country through the College of Health Sciences,
that Sabay University to Japan Public Health Institute through the European Society of Sociology,
Social Works Now Topologies, most recently for some some studies, so that the sort of formal side has changed.
But that doesn't mean that there aren't very serious ethical, dynamic dilemmas about the kinds of things that Santiago was talking about.
If we find out things that are worrying in the research about the safety of our food workers in some circumstances
and about the question of compensation and how we justify what we're doing to the people who spend time with us,
so we give small remuneration or set gifts.
But then you give to some and you don't give to others and people are selected and others are not selected.
There are no perfect solutions and it is constantly a dilemma.
And people do ask, Well, what do I get out of it and why am I doing it?
And then you do get some people who think who really begin to enjoy talking
about their lives and being part of something that they think is interesting,
as we saw from some of the quotes that that Randall put up.
So there are no easy solutions,
and we do do research that isn't like we're giving people a vaccine or something that's going to fundamentally change their lives.
And it's it is uncomfortable and there aren't any obvious solutions.
And we have to I think we have to live with it and try and struggle with explaining it in terms of the institutional versus the individual.
I agree with Randall. I can't help when I think back, it's it's it's some individuals that really mattered.
You know, somebody in UNICEF's that said, let's get together and why don't we start this, this forum that's now been going for 10 years.
It was an individual, a head of a department in the Ministry of Education.
He said, Let's do a project together and you know somebody,
people change an office and you may be able to create some sense in an institution that what we're doing is useful,
but we need people who think their work is important.
I think this work is important really to make a massive difference.
And we do struggle with the changes and we do struggle with trying to search in difficult circumstances.
And there are problems with people, with individuals, with institutions and, you know, political that at times of elections, it's difficult.
It's risky. But somehow being there for a long time and over the time means that, you know,
somehow we build up teams of researchers who we trust and we weed out people that we're
less comfortable with or we don't feel are doing the work as well as we would like.
And so we now have at least in the qualitative team, but also in the deep,
actually in Ethiopia are really people who know about how to do the research well that we can trust and work with.
And I think that's that capacity building aspect has been very important.
But yes,
serendipity in who you meet and what happens and that ability to change the design and approach things in different ways is actually very key.
And finally, to the question of what kind of of the sample size in Ethiopia so.
The young lives, the design worked on the idea of sentinel sites rather than a representative sample, whatever that means.
And there was a pro-poor bias and a food insecure bias.
So more of the rural sites are in the food insecure areas within the four larger regions that were selected and within the urban areas,
more in the poorer areas of urban urban areas.
And as I was saying that, that meant that there was more of a likelihood of those being knocked down.
So you're right, we didn't select Somali region, and I did say in earlier on that if I was redoing this,
I would have taken the risk of trying to do work in some of the emerging agro pastoralist areas because I think that is an important aspect.
But attrition from a statistical point of view and economic surveys is a real risk, and we are already facing that with the older cohort.
And so there was a call taken not to work for this particular survey in the agri pastures areas,
although we did do some school survey work in Somali in Somali region.
But you know that those are the kinds of decisions that have to be made.
And then in my view, every community is representative.
Every community is interesting. Every piece of research is a case.
If a factory is built and is knocking down most of the area or there's a redevelopment,
that in itself is an incredibly interesting thing to to to begin to understand how people cope, how their lives are affected.
And so, yes, of course, it would be nice to do more research in other areas,
but there is an incredible value in the longitudinal approach to following what goes on.
And that's why I would like more of a community focus and understanding the political economy of change.
I'd like to just make a little comment about this issue of ethics, of engaging with respondents over a long period of time,
and obviously you're always challenged to reduce respondent burden as much as you can, but you're also wanting to give back.
This is not, you know, you do not want to be involved in extractive research.
And just to give you an example of a very different view, we were we were evaluated,
reviewed and evaluated by a team some years ago and one of the well,
we had two teams actually evaluating us, and one of the teams was led by statisticians,
and they were really concerned that the engagement that we had,
the reciprocity that we were building up with these communities would contaminate the survey data, and they used the term contamination.
So from the from the point of view of a of statistics and the statistical discipline,
any way in which you're researching that influences people's lives and including
where you're trying to actually give something back and perhaps even more, we you're trying to give something back because you're, you know,
you're not wanting to be extracted that then risks changing the children's lives and changing their trajectories,
which I think is a really interesting and valid point.
Personally, as an anthropologist, I would argue that everything we're doing is impacting their lives.
Just by asking your mother whether she's vaccinated her children, you're going to make her think, Oh, should I be vaccinating my children?
Of course you're having an impact. We try to raise money, actually. Sadly, we didn't get the funding.
We tried to raise money to actually produce a parallel sample to the young life sample in one of our
countries and to actually assess whether or not the study was having an impact in and of itself.
Because I think that some really important research exercise and I would be astonished if we didn't.
And I would just be very saddened if we hadn't in some ways, you know, benefited the children.
However, as I think Santiago was saying, is we are selecting one child in the household,
and we do know from the qualitative research that in some senses, the households in some context think that this is a special child.
They were they were selected for a particular reason. And we worry then about whether, in fact, we've introduced inequality within households.
Is this child getting more investment than the other children in the household, for example?
So I think this is a really, really important issue.
The the this this event is part of a series of methodological learning and lesson learning from young lives and reflection.
One of the one of the outputs from this series will be on research ethics.
These are the kinds of issues that we want to tease out. And I mean, we know that the teams have got extraordinary experience of,
in some cases, really painful circumstances that they've they've been dealing with.
So how do you really deal with that as a researcher without somehow contaminating your data?
Sorry, that's just my hobby horse. Can I ask you any more questions?
Anybody has any further questions and don't put on any communications policy engagement.
We've got quite a lot more into the research research from the Office of the Chief of Staff.
But let's go first then to Robin Naughton from the George Institute for Global Health.
So you alluded to some of the positives of being part of a global group where
sometimes Oxford can say things that people and each of the regions cannot say.
I'd be interested to get a sense from the team of that global versus local and the positives and negatives.
All of of that. And I I guess asking the question of what do people see as was that a real asset to be part of a global team,
even if you are focussing on some of the local things, but vice versa?
What was some of the negatives of that, perhaps if they exist?
Hi, I'm Rosanna humans. I'm from Brooklyn roots and I used to work at young lives many years ago, and so I've got two questions.
One is very quickly about Vietnam. What was Vietnam's experience like when it came to impact and engaging government?
And the second question is a bit more personal.
So what does one do when one has just one three year grant and is quite aware of the area, but not really the country?
We want is working with how exactly do you set about building these relationships within three years with government agencies?
That one really doesn't know anything about and then create some meaningful impact.
Hello, my name is Emma Feeney. I'm also from the George Institute for Global Health,
and I have a question about what you described as impact ology and tax return and the the issue you highlighted
about research centres being put under pressure to make false promises in terms of the impact they can deliver.
Clearly, young lives has had some very nice instrumental impact, which you've got a straight edge today.
But you reference the fact that actually a lot of the focus is on conceptual impact.
So I wondered if you had come up with a way of measuring conceptual impact that you feel
is both valid and is also acceptable to donors that you might want to share with us?
I wish it was addressed to you. I use that. OK? You can tell us how we can, really.
Our northern partner is buck. So I mean, I can I can respond to a few of these questions.
I think the global versus the local. I think we in my mind, it's because we felt the real,
tangible impact of young lives was especially the instrumental and the really important impact was going to be at the country level.
That's why we hired country directors. I don't know why we didn't realise that we needed country directors at the very beginning.
But these guys have made a massive impact on the study in terms of the credibility of the authority,
the prestige, everything that they bring to the study.
We selected with great care and actually each one of them brings something very different to the table.
You know, Brennan perhaps has more experience in policy and the other two have been more involved, perhaps in research.
But in each case, they brought something really unique and important to the study.
So from my perspective, it's the we. The important thing is that the country directors are part of the governance of young lives.
So sometimes decisions get made globally in Oxford on behalf of everybody, and sometimes they get made locally.
But it all needs to be done in collaboration and negotiation and so on.
I have no idea whether everybody in the team thinks it's perfect, but that's how we've tried to manage it for Vietnam.
I don't know. We have a guy from Vietnam is actually with us in the room,
and I don't know whether Zook would would be willing or interested in in answering that question.
I can certainly say that one of the things about Vietnam, which puts it in a slightly different category from the other three countries,
is that our main research partner at the Centre for Economic Forecasting,
CAF Vass, the Vietnamese Academy of Social Sciences, is a government institution,
and our country director for Vietnam is actually one of the lead economists in the country.
And he is the reason he's not here is because he's in so much demand.
He's doing such important work in country at the moment that he couldn't come to Oxford during this week.
So in a way, in Vietnam, we have a route straight into government at the most senior central level.
Each one of our country directors has emphasised the importance of being able
to be agile and work across different ministries and institutions and so on.
But it's incredibly labour intensive, which if you can just produce a report for the Communist Party,
for example, of a central government in Vietnam, then you're straight in at the deep end.
And I think that's one of the big advantages that we have there.
But on the other hand, it isn't as if in Vietnam media is a big opportunity really for for engagement.
So it's a very different style and approach that we would have in the other other countries.
But they're moving towards the the issue of raising false promises, and I'm sure everybody has lots to say about this.
I mean. I think it's just really important to have the donors on board from the outset.
And I think for them to be realistic and pragmatic as well.
I think all the donors that we've ever worked with, people who really, really believe in evidence being a very important part of policy development.
I mean, there are donors who don't actually recognise that and who are much more norms based in their approach.
I think our donors have already self-selected themselves, and they appreciate that research is one step removed.
It's not like intervening directly and running an NGO that actually implements things and so on.
In terms of how we measure, we struggle with the indicators all the time.
We've got some really boring quantitative indicators, like the number of policy briefs or the number of stakeholder events and so on.
I mean that just inputs indicators. They measure nothing, really, as far as I'm concerned.
They just tell you about the volume of activity and maybe the amount of money and energy that's gone into it.
Much more meaningful is to know rates, I think, particularly the conceptual impact, and we do go back quite a lot to people.
In fact, our communications team are often very, very demanding and say, Well,
we have a we have a say, we're having a cup of coffee and somebody says,
Oh, I was really interested in that in that statement that Young Lives Matter and I really thought that China was really, really important.
So the communications team says, Well, get it on paper, get it, get them to, you know, to agree to actually say that in public and so on.
So a lot of it is through qualitative evidence of that nature.
We do develop impact case studies, which we put onto the website sort of showing how things have evolved.
But finally, I just wanted to say this other thing that we've mentioned.
I think the idea of going in cold to a country starting from afresh without all those pre-existing relationships.
I think the first thing you do is a scoping study to see who is who, the who, the people who the players are that are already out there.
We're a small team. We're a very small research team. We need to work with multipliers.
We need to work with intermediaries. We need to work with people who've got reach that we don't have.
And that's been one of the most exciting things that we haven't really discussed is the extent to which our work gets commissioned by,
say, UNESCO's, UNICEF, Oxfam or a whole range of Save the Children lots of agencies, commissioners to do studies.
They are actually much better placed than we are to disseminate our work in many ways because they work globally and they work locally.
They work in many more countries than we do, and they are out there doing advocacy all the time.
So we shouldn't make it look as if we're claiming to be doing it to our own by any means,
partnerships that we're in the Coalition to End Child Poverty, for example, which has got 25 international organisations involved.
So we often provide the evidence that they then carry forward. So I think it's about partnerships, strategic partnerships with with multipliers.
I think I just wanted to comment on the global versus local. I think sometimes it can be a big advantage.
And especially the Oxford brand really works with the same need the AOG at times.
At times you have to just pretend you're an Indian research organisation, so I think you have to play to the gallery.
That's depending on dependent very much on what the flavour of the month is.
So I've been trying to be candid here, but that's how it is.
You have to, you know?
And the Oxford brand does definitely, in many cases, really add, you know, so I'm saying Oxford, not just young lives, because yes,
they understand it's a longitudinal study, but when they hear that, OK,
so the the key technical team sits in Oxford, I think that really adds value to the work we do.
And there is then there is a benchmarking and they think there is quality assurance in terms of the work we do in terms of,
you know, how would you do this in three years? I think it's really difficult, Rosanna,
to say that in three years when you have to show impact and choose the right partners and get a seat on the table,
it's a really hard task and you have you said scoping. But you know, sometimes coping, especially in an unknown territory, is hard.
And I I mean, I know the only way to do it is if you have somebody you know who will then know other
people who will bring that sort of the influence that you need and whether it is,
you know, the research or the policy influencing because you need different skillsets,
I think in the team and just bringing putting that team together is not easy and it's not about qualifications,
it's about the experience they bring to the table,
I think and and the relationships people have, I think that is really important and often we will know if it's.
At a distance. So, yeah, I think that's it.
You have to ask. Yes, I think in terms of the sort of conceptual impact, there are two things that maybe are counterintuitive,
but I think are quite important in terms of a paradigm shift.
And the first one is that there is this tendency to think of things in terms of rather linear theories of change and love frames, et cetera.
Whereas actually things are the reality on the ground is much more unpredictable, and it actually has to be much more collaborative.
If it's not a question of one particular piece of evidence leading to a change.
It's it's accumulation. It's it's it's it's the joint and darkness and also the power relations with
those who can make decisions who can influence things that really matters.
It's what happens behind the scenes. So and there is resistance, of course, and it's something that is gradual and there's a lot of serendipity.
There's a lot of who you meet when and who you talk to and who talks to who.
And so, you know, trying to trace how that happens is not a very simple thing.
So that so that in so far, as research projects can build in an element of unpredictability and an element of letting
things be iterative in the way that things happen and way of joining up with other actors,
that is particularly important. So let me give you an example.
So we were we had a grant from the Child Investment Fund Foundation to do some work on early learning,
and it was fairly open and it was very lucky that we happened to come start the project just at the time when the cycle
of deception and the Education Sector Development Plan was being thought out and we knew we had it was predetermined.
We were going to work on early learning, but it was still not even sure whether we would work on health or education.
And it so happens that the Ministry of Education was moving and the Ministry of Health was not.
And they suddenly were expanding. The amount of early learning wanted to move from five percent to 50 percent over five years.
And it so happened that in the team, Martin would head and those and others had expertise from other countries,
and the European government was open to hearing about that.
But then it was a series of open ended, an iterative approach.
So the first thing we found out was the Ministry of Education had no idea what was happening in the regions.
There were experts in the federal ministry in Addis who didn't know what the regions were doing.
So going out with those experts to see what different regions were doing, which included Somalia and Benishangul, by the way.
So we went to seven different regions where different things were happening.
Some good, some bad, some experimental, some too rapid, some too slow.
So really going with them and seeing what was happening. And then when we came back with findings, people said, Well, there is this oh class,
this one year of school readiness programme that's happening in different ways,
but nobody thought through what the colleges of teacher education are doing.
So that meant going to the the the the colleges of teacher education.
And then people said, Well, you know, what's the demand side, whatever what our community's doing about this?
So another element that was not in the plan was included.
So all I'm saying really is trying to build in a degree of unpredictability and degree of
iteration and a degree of opportunity to respond to what seems to be the important next step.
Rather than coming with your already planned log frames,
theories of change and assumptions of the research is going to go from A to Z in a very linear fashion.
But then funders are not usually very open about this kind of thing.
Very, very briefly. I don't think we have ever used sort of the Oxford argument at Oxford says.
I don't think we have ever used that. But of course, it gives prestige to the project that it comes from here.
And I remember once Len Pritchett, a professor from Harvard, came to Peru and started talking to how about skills,
apparently in Vietnam and peruse, and somebody asked about Vietnam were similar in pre-school.
And then, by the age of 15, children in Vietnam were much ahead.
And so people, so now it was Oxford and Harvard saying something about this,
that some people started asking about this to our country and some journalists came and said, I remember in the Latin American evaluation.
Cuba was was achieving a high level, so they were saying maybe Cuba, Vietnam, the political system,
this is what so we try to keep them away from very simplistic types of explanations.
But I remember those kinds of things sometimes attract attention and and it's good for that.
And I think the other countries are not to anything. My colleagues have said so.
OK, so are there any more questions? All right, so I think maybe we can we can finish.
I would like to just reflect on on a few things from because we have this long term insight,
I suppose, into this process of of, you know, generating impact and so on.
I think it's probably worth saying that. This there was I think it was Kathy Silver who was asking was saying about the
the funding implications of the versatility that you need to you need to have.
So I mean, let's be honest, young lives has been well funded in some respects,
but it's never had the kind of money that was needed to be able to really be responsive in the ways that that everybody's been saying.
We have had to learn to be permanent fundraisers we never stop.
So even though we had the core funding, we've never had the luxury of just being able to sit there and and say, Well, OK, that's it.
Job done because there is this, there's always got to be the ability to be to be agile,
and that always takes more resources than we allow for because we always plan.
I mean, in a longitudinal study, you do plan things well, well ahead, but you can't always change your direction without additional funds.
So I think it's a requirement, sadly, for researchers who are not necessarily wanting to use their time on fundraising.
And it does really does take away time. And I can't even begin to tell you the number of weeks of my time of being invested in fundraising.
I think the other advantage that longitudinal research has over cross sectional research is that with every round,
the evidence becomes more compelling. The power grows.
And I think that this long term relationships of trust that you build to create the enabling environment with your policymakers,
that is also matched by the fact that your data become more and more powerful with time.
So I think it was runner who made that point that actually,
I'm not really sure why we had all these poor policy and communications staff working so
hard at the very beginning of young lives trying to show that we were having impact.
Actually, now I would say to people, don't do that in the beginning starts by building the profile of your work,
start by building your, you know, getting out there and establishing your reputation.
This is our team. This is what we bring to the table. This is what we this is who we're talking to.
This is what we're we. This is what we're hoping for and so on.
And then, you know, build into the impact.
You know, the communications and policy impact side of it will just grow at a later stage when you're ready for it because it's so frustrating.
If you've only got one round of data or even just two rounds of data, there's not an awful lot you can say about trends.
So I think that's that's another thing. And I think the other thing I've learnt is that it is really important to involve your donors really closely,
have them on your advisory board, have them really have the ownership that's needed of the research.
So they get it.
So they understand why you have struggles, why you can't always meet your objectives, why are you being forced to shift gears and so on and so forth.
So I think I think that relationship should not be underestimated.
I think Al-Ula, who has made it very clear that he has a problem with some of these new templates and so on.
Now that the impact agenda has risen so high, it's right.
If it's public money, if it's if you know it's right that we're we're relevant and and have actually
got a role as researchers in the world of policy and practise and so on.
But then is it going to be achieved through these templates and so on?
And I think he's right to be critical of them.
But actually just in defence of the poor old log frame that's so hated by everybody and even the theory of change,
these should be seen as organic working documents. Ours actually is changed regularly every single year it's revisited with with the donors and so on.
We say, Look, this was unrealistic. This is we're going in a different direction.
We're going to have to shift this. They should not be seen as set in stone and we should not feel constrained to work just to that.
I think that's really important. But there are tools and they should be treated organically.
But I think Alan is also right to point out there's an awful lot more going on there that
you're not necessarily recording in these rather narrow templates and showcase that and be,
you know, be the kind of fight back if you feel that you're, you know, you're not being, you're not.
That research impacts are not sufficiently valued as compared to, say, intervening directly and so on.
Some pathologies that went over that they impact ology as the word of the day.
Thank you, everybody, very much for joining us.