1
00:00:00,510 --> 00:00:05,520
Thank you. Good afternoon, everyone, and thank you for coming.

2
00:00:05,520 --> 00:00:10,680
I'm Margaret Stephens, I'm head of the economics department and I'm very honoured to have been invited

3
00:00:10,680 --> 00:00:17,430
by the trustees of the Sanjay Allow Memorial Trust to chair this event.

4
00:00:17,430 --> 00:00:28,050
I want to say a few words about both the trust and also the panellists very distinguished panel that we have here this evening.

5
00:00:28,050 --> 00:00:34,290
First of all, just to give you the background to this event and the centennial, our trust,

6
00:00:34,290 --> 00:00:41,850
it was founded in honour of to honour the memory of Sanjay Alal and his work and it,

7
00:00:41,850 --> 00:00:46,770
together with Green Templeton College Green College was was his college.

8
00:00:46,770 --> 00:00:52,590
It enables the university to bring to Oxford each year for one term,

9
00:00:52,590 --> 00:01:00,960
a distinguished visiting professor in the WHO works in the area of business and development.

10
00:01:00,960 --> 00:01:05,250
Sanjay Allow came to Oxford in the early sixties.

11
00:01:05,250 --> 00:01:11,730
He was a student at the beginning and apart from a few years at the World Bank,

12
00:01:11,730 --> 00:01:18,720
he spent the whole of his career as a development economist and researcher here in Oxford.

13
00:01:18,720 --> 00:01:22,350
He was initially at the Institute for Economics of Statistics.

14
00:01:22,350 --> 00:01:31,680
He went to Queen Elizabeth House, where he became professor of Development Economics and a fellow of Green College.

15
00:01:31,680 --> 00:01:36,990
And some of you, I'm sure, are familiar with his work for those who aren't.

16
00:01:36,990 --> 00:01:42,000
He was a prolific writer on all aspects of development, especially.

17
00:01:42,000 --> 00:01:48,030
He was particularly interested in technology transfer and the positive role that governments,

18
00:01:48,030 --> 00:01:56,700
national governments can play in terms of policy development policy to facilitate that.

19
00:01:56,700 --> 00:01:59,040
He was an advisor to too many governments.

20
00:01:59,040 --> 00:02:07,380
He had a formidable international reputation as as an advisor to governments and international organisations.

21
00:02:07,380 --> 00:02:09,690
I knew him only a little.

22
00:02:09,690 --> 00:02:19,710
We overlapped when I first came to Oxford, so I'm going to turn to to to make Ned, Deci said about him in the obituary in 2005.

23
00:02:19,710 --> 00:02:27,480
He said unusually for an academic economist, he overcame the characteristic culture clashes between social scientists and scientists,

24
00:02:27,480 --> 00:02:33,240
economists and engineers, economists and business managers, academics and practitioners.

25
00:02:33,240 --> 00:02:43,110
He made it his business to know how things actually worked on the ground and not how they might be assumed to work from the comfort of an armchair.

26
00:02:43,110 --> 00:02:52,830
And so it does seem to me that he would have particularly approved of the 2018 Sanjay Allow visiting professor Esther Duflo,

27
00:02:52,830 --> 00:03:01,390
whom we welcome to Oxford today. She has just arrived as the 2018 visiting professor.

28
00:03:01,390 --> 00:03:04,900
Ronnie asked me to say also that she is very,

29
00:03:04,900 --> 00:03:14,500
very proud as a as a trustee and really organiser of this event to the female line-up we have this evening.

30
00:03:14,500 --> 00:03:29,910
There is. So a first female head of the economics department, a first, Sanjay, allow a visiting professor,

31
00:03:29,910 --> 00:03:36,280
and we can probably come up with a few other firsts as well amongst us.

32
00:03:36,280 --> 00:03:43,080
Esther is the Abdul Latif Jameel, professor of poverty alleviation and development economics at MIT.

33
00:03:43,080 --> 00:03:51,960
And I am sure many people here are familiar with her career and many accolades to date and the impact that she's had on development economics.

34
00:03:51,960 --> 00:04:00,720
She co-founded and co-directs the outstandingly successful Poverty Action Lab Jay Jay Powell at MIT,

35
00:04:00,720 --> 00:04:10,560
which is established randomised controlled trials as a primary tool of policy of evaluation, particularly in development context.

36
00:04:10,560 --> 00:04:15,890
Her focus is on what evidence can tell us about how to improve the lives of the poor.

37
00:04:15,890 --> 00:04:22,620
And her book, which some of you I'm sure have read poor economics in with Abhijit Banerjee,

38
00:04:22,620 --> 00:04:31,620
they argue that there are no universal answers to the question, and it really depends on the context in which a policy is applied.

39
00:04:31,620 --> 00:04:38,550
To quote, they say that requires us to step out of the office and look more carefully at the world.

40
00:04:38,550 --> 00:04:43,920
And that's why I think Sanjay allow her to very much approved of what they have been doing.

41
00:04:43,920 --> 00:04:48,180
We also have Rachel Glenister here, who's come since the beginning of this year.

42
00:04:48,180 --> 00:04:53,220
She's been chief economist at the UK Department for International Development.

43
00:04:53,220 --> 00:05:04,230
Like some JLL, she started as a as an Oxford post, but she's since worked as an economic advisor at the Treasury, at the IMF and the World Bank,

44
00:05:04,230 --> 00:05:17,860
before also being drawn into the orbit of Jay Powell, where she was the executive director from 2004 until arriving in the UK this year.

45
00:05:17,860 --> 00:05:24,190
She's worked on evaluation on policy evaluation in the areas of microfinance and governments,

46
00:05:24,190 --> 00:05:31,570
governance, health and education, and sometimes indeed with with Duflo and Banerjee.

47
00:05:31,570 --> 00:05:36,190
And so she's really ideally placed this evening to provide her own perspective,

48
00:05:36,190 --> 00:05:41,830
her own evaluation, if you like, of the role of actors in policymaking.

49
00:05:41,830 --> 00:05:45,610
And finally, I didn't really mean to leave him out earlier.

50
00:05:45,610 --> 00:05:52,330
Sir Richard Peto is the distinguished professor of medical statistics and epidemiology at this university,

51
00:05:52,330 --> 00:05:57,400
and he's a trustee of the Sanjay Allowed Trust. And like Sanjay, I allow.

52
00:05:57,400 --> 00:06:03,130
He spent almost all his career in Oxford, and in parallel with Esther,

53
00:06:03,130 --> 00:06:11,680
he founded and co-directs the Oxford Clinical Trial Service Unit and Epidemiological Studies Unit.

54
00:06:11,680 --> 00:06:18,580
It doesn't have quite such a catchy name as the Poverty Action Lab, but it conducts large,

55
00:06:18,580 --> 00:06:26,140
randomised trials and meta analysis what worldwide, according to the Royal Society.

56
00:06:26,140 --> 00:06:31,750
Richard has shaped the way modern clinical trials and population studies are carried out worldwide.

57
00:06:31,750 --> 00:06:40,840
His own research has saved countless lives by quantifying and highlighting the preventable causes of cancer and cardiovascular disease,

58
00:06:40,840 --> 00:06:45,760
smoking in particular, and by identifying optimum treatment choices for both.

59
00:06:45,760 --> 00:06:54,010
So he brings a different perspective to our discussion this evening and to randomised controlled trials in particular.

60
00:06:54,010 --> 00:07:00,310
In another link with Esther and Rachel's work, he has recently studied malaria in Africa and India.

61
00:07:00,310 --> 00:07:14,500
So please welcome all of our panellists. So we have an ambitious title.

62
00:07:14,500 --> 00:07:28,760
Our focus is on education. This evening, I'm going to invite Esther to speak first.

63
00:07:28,760 --> 00:07:36,610
And I thought it's first, so first of all, think, let me start by expressing my gratitude,

64
00:07:36,610 --> 00:07:46,660
thinking you will very warmly for me here today, not just to to give a talk, but to be the Sun lay professor.

65
00:07:46,660 --> 00:07:58,720
As Mike pointed out, this is just getting this name is in itself an honour especially far for me,

66
00:07:58,720 --> 00:08:08,470
given the type of work I do and the ambition that this work has to make a difference in the world while being academically relevant.

67
00:08:08,470 --> 00:08:12,820
So we are, you know, this is one of the old model that we are trying to follow,

68
00:08:12,820 --> 00:08:20,320
and this is a wonderful opportunity to to, I guess, to reflect on that.

69
00:08:20,320 --> 00:08:28,810
Another thing, of course, is this institution is all I think can already be called an institution for how

70
00:08:28,810 --> 00:08:38,080
successful it has been in getting a hugely distinguished line up of professors before me,

71
00:08:38,080 --> 00:08:43,180
and I'm even more honoured to to follow in their footsteps.

72
00:08:43,180 --> 00:08:47,830
And finally, I'm delighted to be here today with the two of you.

73
00:08:47,830 --> 00:08:57,250
I should add one first that I think is relevant, which is actually the first female chief economist of Difford.

74
00:08:57,250 --> 00:09:01,690
So wonderful. She also, I suppose, is the first female executive director of Japan.

75
00:09:01,690 --> 00:09:06,400
But that's less interesting because she's the first executive dogs on the effects of that.

76
00:09:06,400 --> 00:09:11,650
So there was not a long line of males before her.

77
00:09:11,650 --> 00:09:20,650
So what? I am going to try to talk about today is really one illustration of the way

78
00:09:20,650 --> 00:09:29,780
that we attempt to to address of two attack issues in Japan and in particular,

79
00:09:29,780 --> 00:09:39,820
all the way. I've tried to do it in my work, which is some interplay between trying to understand the nature of our policy questions.

80
00:09:39,820 --> 00:09:50,620
So here is the one slice of the problem of education in the world and understand them, not just to understand them or do that.

81
00:09:50,620 --> 00:10:03,820
That's quite nice, but in order to to have enough things to say about what could be done about the problem and then try

82
00:10:03,820 --> 00:10:14,650
and go one more step and work with government to try and put this possible solutions into practise.

83
00:10:14,650 --> 00:10:20,710
So education is a good example to do that. First of all, it's a very important and interesting problem.

84
00:10:20,710 --> 00:10:24,070
So going off, we you to read a bit more mature in a sense.

85
00:10:24,070 --> 00:10:36,040
We've covered it's one where we have had to tend to cover that entire route and not just do the research of it, such other things like that do.

86
00:10:36,040 --> 00:10:40,790
There are other domains like that, but it's maybe the most complete.

87
00:10:40,790 --> 00:10:49,280
So what's the status of the world's primary education in three interior bullets, which which is, of course,

88
00:10:49,280 --> 00:10:55,970
a somewhat of a caricature, but it's worth the kind of setting the stage, the unemployment rate.

89
00:10:55,970 --> 00:11:00,110
So I'm going to talk to them mostly about primary school, almost exclusively about primary school,

90
00:11:00,110 --> 00:11:05,540
with a little bit of mention of pre, primary and post-primary.

91
00:11:05,540 --> 00:11:11,900
So we're thinking about primary education today, and all rates are very high worldwide.

92
00:11:11,900 --> 00:11:15,560
So in a sense, that battle is being won.

93
00:11:15,560 --> 00:11:24,560
This is not the issue, except in some countries or some extraordinary circumstances is not to get kids to school, even gold.

94
00:11:24,560 --> 00:11:29,840
The issue is to teach them something when they come in because they are on board.

95
00:11:29,840 --> 00:11:35,570
The attendance rates are not that high and the learning levels are very low.

96
00:11:35,570 --> 00:11:42,470
So let me elaborate on this three points quickly. So this is a map of the world with enrolment rates.

97
00:11:42,470 --> 00:11:48,920
Unfortunately, the the I took it from the World Bank and it didn't see fit to put the key.

98
00:11:48,920 --> 00:11:56,330
But the the dark red means above 90 percent enrolment, and you can see that basically the whole world is dark red.

99
00:11:56,330 --> 00:12:04,730
And if we're looking at the low income countries here, the share of kids who are enrolled in primary school is actually just at 100 percent.

100
00:12:04,730 --> 00:12:14,030
If you're wondering why they can be 100 more than 100 percent and all, that's the the way they are kids who are in primary school, who shouldn't be.

101
00:12:14,030 --> 00:12:22,340
So we take the population in primary school and divide by the fraction of kids of primary school age, and typically you get a number above 100.

102
00:12:22,340 --> 00:12:24,350
So of course, much lower doesn't matter.

103
00:12:24,350 --> 00:12:33,340
You just have to know what this is much lower in pre-primary, much lower and secondary and tertiary, but even that is really increasing a lot as well.

104
00:12:33,340 --> 00:12:43,090
And you can see that the difference between boys and girls is really a very small.

105
00:12:43,090 --> 00:12:49,600
If we are looking at India in particular, which a lot of the examples I'm going to talk about today are going to focus on India,

106
00:12:49,600 --> 00:12:53,710
and then I think Rachel is going to expand to the rest of the world.

107
00:12:53,710 --> 00:12:57,880
We have the same finding where all the and all men rates are very high.

108
00:12:57,880 --> 00:13:01,090
So the net and all men register. I'm asking every child, are you?

109
00:13:01,090 --> 00:13:06,130
And this in school is almost ninety seven percent.

110
00:13:06,130 --> 00:13:13,240
The attendance is much lower. The attendance on any random day in September is just seventy one percent.

111
00:13:13,240 --> 00:13:20,860
So many kids are absent for whatever reason. And you can see some differences across states.

112
00:13:20,860 --> 00:13:30,370
That's a problem, but the bigger problem is really not there. The problem is that while kids are in school, they don't learn very much in school.

113
00:13:30,370 --> 00:13:38,080
So in India, it has been documented for years now by the organisation Pratham that you're going to hear a lot about today,

114
00:13:38,080 --> 00:13:43,570
which is a wonderful movement on education in in India.

115
00:13:43,570 --> 00:13:49,480
And one of the things they have been doing since the early 2010s is, in fact,

116
00:13:49,480 --> 00:13:58,630
they started in 2005 is a survey of all of a representative sample of kids in every single district in India.

117
00:13:58,630 --> 00:14:01,960
So it's a very large sample of kids and they're going to their home.

118
00:14:01,960 --> 00:14:07,750
So it's irrelevant with all the kids in private school or public school or not in the world, et cetera.

119
00:14:07,750 --> 00:14:16,990
And what they find is that for kids who are in standard five, that's the last grade of primary school are less than half of them are able

120
00:14:16,990 --> 00:14:22,780
to read a simple fog off which they should be able to master by stand up to.

121
00:14:22,780 --> 00:14:31,930
And even if you're looking at one level or letters level, the the achievements are correspondingly low.

122
00:14:31,930 --> 00:14:43,480
The second thing that is striking in this picture is that this started in 2005 would have put the at the club since 2011 is that there is no progress.

123
00:14:43,480 --> 00:14:48,130
In fact, in 2016, they got depressed and they stopped doing.

124
00:14:48,130 --> 00:14:52,360
In 2017, they didn't do one. And in 2018 they did second high score.

125
00:14:52,360 --> 00:15:01,870
So we don't know what's happening after that. But from 2005 to 2017, there was no new change in that number.

126
00:15:01,870 --> 00:15:06,040
This is the cost. They also do a simple mathematics test.

127
00:15:06,040 --> 00:15:09,940
So if you are depressed about the reading test, the mathematics test is somewhat worse,

128
00:15:09,940 --> 00:15:16,030
so they have a simple subtraction, which carry over two digit subtraction with carryover.

129
00:15:16,030 --> 00:15:23,020
And you can see that in standard, five 50 percent of kids are able to do that subtraction.

130
00:15:23,020 --> 00:15:26,350
And then they have a division and you stand up five.

131
00:15:26,350 --> 00:15:34,240
Twenty six percent are able to do that division, which is just one number divided by one by a single digit number.

132
00:15:34,240 --> 00:15:43,240
So the achievement in mathematics are even even lower. So this is not just an India problem, although it's better.

133
00:15:43,240 --> 00:15:47,030
It's been documented in the deeper and more systematic way in India.

134
00:15:47,030 --> 00:15:56,320
They'll know it's similar, always easier for other countries or for other countries which find very, very similar things.

135
00:15:56,320 --> 00:16:05,140
A lot of kids go to school, they don't attend very well. Usually they learn very little or a lot of them done very little or nothing.

136
00:16:05,140 --> 00:16:15,730
So of course, given that issue, people have been thinking about what this might be due to, and there are lots of everyone has a favourable hypothesis.

137
00:16:15,730 --> 00:16:22,240
What I want to stop trying to rule out is what in my view, not just my view,

138
00:16:22,240 --> 00:16:31,300
but maybe it's not a consensus, but maybe to some extent, at least what the problem is not due to.

139
00:16:31,300 --> 00:16:38,590
So I want to rule out some of the explanations that are in in the home.

140
00:16:38,590 --> 00:16:44,620
But before that, I want to talk. I want to go one step deeper than just this diagnostic.

141
00:16:44,620 --> 00:16:46,960
This this set of slides.

142
00:16:46,960 --> 00:16:56,830
Oh, very almost like a you know, everybody says everybody says, and it is a problem that all the kids go to school and a little nothing.

143
00:16:56,830 --> 00:17:03,280
But I think it is worth saying it is worth realising that there is an even deeper problem.

144
00:17:03,280 --> 00:17:18,520
Which is that not only the kids that the kids are learning nothing but the schools actually unable to identify what the kids know anyways.

145
00:17:18,520 --> 00:17:20,380
So you can say, well, look, kids are learning nothing at school,

146
00:17:20,380 --> 00:17:25,240
but at least the school would fund some kind of certification rule or something like that.

147
00:17:25,240 --> 00:17:39,290
And what we. But what is finding on this very scholarly tasks of doing division and reading reflect is also the fact that children.

148
00:17:39,290 --> 00:17:46,500
Existing knowledge is not picked up by the score and if anything is wasted and destroyed by the schools.

149
00:17:46,500 --> 00:17:52,030
So let me try to talk to be more precise on that.

150
00:17:52,030 --> 00:17:56,200
So the first thing I'm going to show you is that you can tell small kids,

151
00:17:56,200 --> 00:18:04,480
I'm going to describe a little bit an experiment that we did with a team of cognitive psychologists at Harvard,

152
00:18:04,480 --> 00:18:12,040
which goes by the term of baby lab at Harvard, led by Liz Pelke in pre-school.

153
00:18:12,040 --> 00:18:18,010
And I'm going to show you that you can take Indian pre-school and teach them a lot of mathematics.

154
00:18:18,010 --> 00:18:24,340
But once they go to formal school, the school system is unable to build on it.

155
00:18:24,340 --> 00:18:30,880
That's the first thing I want to do. And the second thing is, look, you know, this is going to mirror my pre-school for me.

156
00:18:30,880 --> 00:18:37,750
And then my middle school phobia is going to look at slightly older kids who are working in markets.

157
00:18:37,750 --> 00:18:45,130
I'm sure that the schools can actually do complex mathematics, not complex with metrics, but very quickly, very good at it.

158
00:18:45,130 --> 00:18:50,950
And yet they are terrible at school maths, even though many of them are actually and old in school.

159
00:18:50,950 --> 00:18:57,640
So let me spend a little bit of time on this. So the pre-school mathematician is an experiment that actually a series of experiment.

160
00:18:57,640 --> 00:19:01,520
We are about to embark in the sound lab to get us a deal.

161
00:19:01,520 --> 00:19:08,920
Not the first one was one with about 500 kids in the in preschoolers in Delhi.

162
00:19:08,920 --> 00:19:14,350
And we we they were all in this little informal preschools in slums,

163
00:19:14,350 --> 00:19:21,640
and we randomised the pre-school to three one of three conditions a mass games where I'm going to tell you what mass games were.

164
00:19:21,640 --> 00:19:29,740
But they were like a curriculum of a few weeks of abstract games to teach mathematics social games,

165
00:19:29,740 --> 00:19:33,790
which is very similar games, but to teach social skills.

166
00:19:33,790 --> 00:19:39,130
The main purpose of that was to have an active control where there is the same kind of game play introduced in the school,

167
00:19:39,130 --> 00:19:42,760
but it's not maths and the sandwich was just the normal curriculum.

168
00:19:42,760 --> 00:19:51,340
So no, no change. So we we started with a pre-test, then we run the game for three months and then we added post-GST.

169
00:19:51,340 --> 00:19:59,740
And then we add a second post test and then a second test until push test after the kids have joined school.

170
00:19:59,740 --> 00:20:02,110
So this is an example of the type of things that we do.

171
00:20:02,110 --> 00:20:08,590
And so this school spits off doing so you need to tell which has more dots, red dots or blue dots.

172
00:20:08,590 --> 00:20:16,600
The interesting thing about our learning of mathematics is that every child in every culture

173
00:20:16,600 --> 00:20:25,060
battling extreme disability is able to do this type of exercises despite no exposure to maths.

174
00:20:25,060 --> 00:20:29,710
And then so if you know that this one so then needed to do these things.

175
00:20:29,710 --> 00:20:38,910
The second thing that was known before we did the study is that this in principle can be trained and it has been trained in the lab in small,

176
00:20:38,910 --> 00:20:41,470
small, a very small scale.

177
00:20:41,470 --> 00:20:51,550
The self thing that was not known is whether or not, if you did that, you could actually help the learning of formal mathematics in school.

178
00:20:51,550 --> 00:20:56,920
So once they have a softer account, they can turn it back and know that in fact, they were correct.

179
00:20:56,920 --> 00:21:02,830
They are more red dot. In fact, they are 10 red dot and five blue dots.

180
00:21:02,830 --> 00:21:07,120
Similarly, we did. The visual form analysis is also something kids can do.

181
00:21:07,120 --> 00:21:15,430
They have to turn the right card. This one is the wrong card to it doesn't have a smiley is winning the fight card, so he does a smiley.

182
00:21:15,430 --> 00:21:19,780
This where the social games very, very similar exercises.

183
00:21:19,780 --> 00:21:29,530
Except you can see in the software, you can see who is happier and the girl or the blue girl and then which fish doesn't go with the rest.

184
00:21:29,530 --> 00:21:35,080
So the idea is the rules were exactly the same. The gameplay was exactly the same, but it didn't rely on mathematics.

185
00:21:35,080 --> 00:21:39,840
It relied on on social skills.

186
00:21:39,840 --> 00:21:49,470
The important findings that we find is that at the first in line, we saw that kids are much better at doing this kind of non symbolic mass.

187
00:21:49,470 --> 00:21:58,350
So in fact, we were good at training them and even better also at symbolic but symbolic much we

188
00:21:58,350 --> 00:22:03,150
can see that they are better at point to no as well so early on in the experiment.

189
00:22:03,150 --> 00:22:07,290
The experiment was effective at teaching them mathematics,

190
00:22:07,290 --> 00:22:14,220
both in the non symbolic mathematics and symbolic mathematics, even though there was no symbol in the experiment.

191
00:22:14,220 --> 00:22:16,710
The effects are very persistent on non symbolic.

192
00:22:16,710 --> 00:22:24,780
Mathematics are very persistent over time, which is not the case for most of the education experiment where you often see a decay.

193
00:22:24,780 --> 00:22:34,770
In this case, we see at the end line to an online tree, you can see the two bars, the effect persisting through 12 months later, 18 months later.

194
00:22:34,770 --> 00:22:45,510
They still have this extra mathematical mathematical abilities, but there is no transfer to a school maths once they go to school.

195
00:22:45,510 --> 00:22:57,720
So you have kids who have a better ability to do this kind of informal mathematics that we know all the bedrock upon which formal mathematics is run.

196
00:22:57,720 --> 00:23:06,480
But despite that, they are not able to learn formal school mathematics. And the reason is that the way that we surmise is that the ways at school

197
00:23:06,480 --> 00:23:12,070
mathematics are taught in school in India is closer to 2.3 than to mathematics.

198
00:23:12,070 --> 00:23:16,110
You were learning that the table of multiplication and these type of things,

199
00:23:16,110 --> 00:23:21,740
the fact that you have any understanding of maths is sort of neither here nor there.

200
00:23:21,740 --> 00:23:32,310
So, so that's the first experiment. The second, which is not an experiment, a descriptive study of of kids, adolescents,

201
00:23:32,310 --> 00:23:40,890
young adolescents in two markets in Calcutta and in Delhi, where we we did two things.

202
00:23:40,890 --> 00:23:50,430
So first, we went to see the kids in the market and we sent mystery shoppers to to buy two product like,

203
00:23:50,430 --> 00:24:01,720
for example, eggplant and tomatoes, some funny quantities like three of eggplant and seven kind of tomatoes.

204
00:24:01,720 --> 00:24:09,250
And when we give them, say, 500 rupees and ask for the change, so the operation requires to apply the unit,

205
00:24:09,250 --> 00:24:16,000
what to do your petition, to do your auditions and then produce a picture.

206
00:24:16,000 --> 00:24:20,920
So we did that three times. I'll show you the result of that in a second.

207
00:24:20,920 --> 00:24:24,820
And then we pulled out the kids if they were willing and if their parents were

208
00:24:24,820 --> 00:24:31,720
willing to give them more like a set type for middle school mathematics questions.

209
00:24:31,720 --> 00:24:35,710
That's the first thing we did. And then the second thing we did is to do the same thing.

210
00:24:35,710 --> 00:24:40,760
But we schoolchildren are in very similar communities, kids in kids in school.

211
00:24:40,760 --> 00:24:43,600
By the way, a lot of the market kids are also in school.

212
00:24:43,600 --> 00:24:51,430
But then we went to school children who are not working in markets for the most part, and we gave them the formal mathematics.

213
00:24:51,430 --> 00:24:56,890
And we also set up a market like that where they have to buy.

214
00:24:56,890 --> 00:25:04,990
They have to sell items. So in the same way, you can ask for some kilos of vegetable or some you need of beans and stockland.

215
00:25:04,990 --> 00:25:14,080
And what did we find? We find we found that if you look at the the formal school mass mathematic,

216
00:25:14,080 --> 00:25:18,700
you can see that the the market children are much worse than the school children.

217
00:25:18,700 --> 00:25:25,720
So the highest level possible is division level. And we only have a 16 percent of market kids at the division level.

218
00:25:25,720 --> 00:25:29,560
But the fifty seven percent of the school children are division. No.

219
00:25:29,560 --> 00:25:35,050
So the kids, the the market kids are much worse at the formal mathematics.

220
00:25:35,050 --> 00:25:39,010
But if you look at the market transaction,

221
00:25:39,010 --> 00:25:47,770
then it's the opposite where the the market kids performance on the mathematics question of ninety four percent.

222
00:25:47,770 --> 00:25:55,240
Ninety five percent of them. Get it right. This complicated maths. Well, the school children are pretty bad between the 50.

223
00:25:55,240 --> 00:25:59,860
About 50 percent of them. Get it right. And in addition.

224
00:25:59,860 --> 00:26:04,900
And by the way, the school, the kids are better than the adults than the market's adults.

225
00:26:04,900 --> 00:26:16,120
And so the more we left the school children unlimited amount of time in this first study, and you can see how they tried to calculate this.

226
00:26:16,120 --> 00:26:21,480
So even the one who got it hard get you tried about 10 minutes of adding something productive,

227
00:26:21,480 --> 00:26:29,470
something productive, which is that even if they got the right result is not like, what could you do with that?

228
00:26:29,470 --> 00:26:34,180
They couldn't possibly work in a market. So from this study, you learnt two things.

229
00:26:34,180 --> 00:26:37,420
And but one is that the there is we did many,

230
00:26:37,420 --> 00:26:44,740
many more things with the market kids to make sure that it was not just that they knew some things by heart and it's just automatic.

231
00:26:44,740 --> 00:26:48,010
They're changing the prices, et cetera.

232
00:26:48,010 --> 00:26:55,840
And if any breaks down at the moment of abstraction, it breaks down when they try to apply algorithm they belong to school in school,

233
00:26:55,840 --> 00:27:03,610
unless until they use their own methods, they can figure it out even with other prices, even with other quantities.

234
00:27:03,610 --> 00:27:09,550
But if you ask them to bet, if you ask them a formal abstract problem, they try to do what they've learnt in school.

235
00:27:09,550 --> 00:27:14,710
And that's where they get it. They get it wrong. So that's the first thing.

236
00:27:14,710 --> 00:27:19,330
So the school system basically destroys the knowledge that exists in these kids.

237
00:27:19,330 --> 00:27:26,950
And the second thing is that what it teaches to the two kids who are actually in school is and practical.

238
00:27:26,950 --> 00:27:31,210
They can't do anything with it. What are you going to do? You can't rely on third down mathematics.

239
00:27:31,210 --> 00:27:39,160
If your understanding of multiplication is to is to write something, ten times to do 13 times that.

240
00:27:39,160 --> 00:27:46,210
Neither can you walk in a market, so it doesn't impact any skill that is useful and also doesn't grow.

241
00:27:46,210 --> 00:27:52,270
In a sense, this is not a skill that do exist in the population.

242
00:27:52,270 --> 00:27:55,840
So that's why the problem is worse than just the supplement.

243
00:27:55,840 --> 00:28:02,050
It's deeper, and it's it really creates an enormous, an enormous waste.

244
00:28:02,050 --> 00:28:08,280
So what's the problem? I was advertising to you that I would talk about what the problem is not.

245
00:28:08,280 --> 00:28:10,560
So there are three things that you hear commonly,

246
00:28:10,560 --> 00:28:16,470
and that I think it is not the first thing is not it is not because the kids cannot learn all winter long.

247
00:28:16,470 --> 00:28:22,290
You have various versions of this argument. The teachers will tell you that the parents are lazy or the children.

248
00:28:22,290 --> 00:28:32,490
Children are lazy. Bill Gates went to India a few months ago, and he explained that we really need to talk to to improve the nutrition in India.

249
00:28:32,490 --> 00:28:38,940
It's because it's because kids are not properly noise that they can't learn in school.

250
00:28:38,940 --> 00:28:42,360
That's a more politically correct way of saying the kids are stupid.

251
00:28:42,360 --> 00:28:49,200
That's why they don't learn to. That's that kind of a popular explanation in a form or another.

252
00:28:49,200 --> 00:28:55,020
By the way, it's an explanation that is internalised by the internalised by the children very quickly.

253
00:28:55,020 --> 00:29:01,770
Well, and which contributes to worsening the problem because they say, of course, they complain, I'm stupid anyway.

254
00:29:01,770 --> 00:29:07,380
So for example, when we ask our market kids to to do the do the mass operation,

255
00:29:07,380 --> 00:29:12,180
they're like, Look, I've dropped out like, there is the reason I can't do this stuff.

256
00:29:12,180 --> 00:29:19,350
And so that's one I'll elaborate in a minute, but I don't think that's that.

257
00:29:19,350 --> 00:29:24,120
The second thing that I don't think it is is teacher salaries.

258
00:29:24,120 --> 00:29:26,880
It is not because teacher salaries are too low.

259
00:29:26,880 --> 00:29:33,540
In fact, I think there is a real problem of teacher salaries in the US and probably almost created in France.

260
00:29:33,540 --> 00:29:38,100
But it's very different in in developed and developing countries where, on the contrary,

261
00:29:38,100 --> 00:29:44,490
teacher salaries tend to be high quality to the rest of the distribution. So I don't think it's teacher salaries, either.

262
00:29:44,490 --> 00:29:55,030
And finally, I also don't think it's incentive to do the job, at least as teacher understanding that can play a role, but I don't think it's a major.

263
00:29:55,030 --> 00:30:01,180
So let me elaborate on these three things, I believe. So why do I think that kids get along well?

264
00:30:01,180 --> 00:30:03,610
Because we studied them, we've studied them.

265
00:30:03,610 --> 00:30:16,060
If you go back to the pre-school mathematician, we actually have the question of kids in this non symbolic games of our little Indian kids.

266
00:30:16,060 --> 00:30:27,720
And we have the same of kids in the laboratory who are children of graduate students, of an assistant professor in Cambridge, US.

267
00:30:27,720 --> 00:30:34,680
And the progression in the games and understanding of the game is similar in the two contexts.

268
00:30:34,680 --> 00:30:40,140
So it's not that kids can't learn because this kids can log just fine. It is.

269
00:30:40,140 --> 00:30:41,940
It sees the material.

270
00:30:41,940 --> 00:30:49,110
Moreover, we find the same correlations between current and subsequent symbiotic skills and not simply skills that is found in the US,

271
00:30:49,110 --> 00:30:56,730
suggesting that the very basic cognitive processes that we find in the US also exist in India.

272
00:30:56,730 --> 00:31:03,160
So the Indian kids are no different. Therefore, they should be able to learn just as well.

273
00:31:03,160 --> 00:31:13,200
Now, they don't have as much help. They don't have support at home often, but they they are increasing all their.

274
00:31:13,200 --> 00:31:17,700
Did show salaries do, as I said, are highly paid a teacher,

275
00:31:17,700 --> 00:31:26,280
salaries are much higher in public school than in private school and then and yet private schools are accused as of performing.

276
00:31:26,280 --> 00:31:34,560
I'll tell you more about it in a minute. Mom, come in and teach us like contract teachers, which are very common in many countries,

277
00:31:34,560 --> 00:31:39,060
are paid a fraction of teacher salaries and tend to be more more efficient.

278
00:31:39,060 --> 00:31:42,570
And finally, there was a very large school experiment in Indonesia,

279
00:31:42,570 --> 00:31:50,860
which basically doubled the teacher salary and produced no effect on learning whatsoever.

280
00:31:50,860 --> 00:31:59,410
Teacher incentives, so I've done some work on that, and clearly you have some effect at the margin of higher teacher incentive, for example,

281
00:31:59,410 --> 00:32:01,450
if you give them incentive to show up more often,

282
00:32:01,450 --> 00:32:09,010
they do show up more often and they are and therefore the test scores of the kids increase a little bit.

283
00:32:09,010 --> 00:32:16,450
In Andhra Pradesh, there was a study showing that if you give incentive based on test scores increase a little bit,

284
00:32:16,450 --> 00:32:23,890
although in another study in Kenya, they found that there was also a lot of gaming on the test.

285
00:32:23,890 --> 00:32:28,810
But the reason why I think that's not the key problem is because there is a very nice

286
00:32:28,810 --> 00:32:35,760
study by Karthik Monty Don looking at private schools in India as in private schools,

287
00:32:35,760 --> 00:32:40,930
as Tonga's incentive to do what parents expect to be done.

288
00:32:40,930 --> 00:32:51,070
So and when you look at private schools, what your findings were, what did this study give kids scholarships to attend private schools?

289
00:32:51,070 --> 00:32:58,150
And now you can compare kids who got the scholarship who are much more likely to attend private school to kids who didn't get the scholarship.

290
00:32:58,150 --> 00:33:04,360
Well, not very likely to have attended private school. And those who are randomised to they are exactly comparable or those of study we

291
00:33:04,360 --> 00:33:09,270
had on private school can confuse the selection effect that kids who are brighter,

292
00:33:09,270 --> 00:33:14,380
whose parents are more motivated are more likely to be in private school. But this has none of that.

293
00:33:14,380 --> 00:33:22,030
And what you find is that private schools are much better at teaching English, but public schools make no attempt to teach English.

294
00:33:22,030 --> 00:33:32,210
So that's not the that's that's not fair composer. But there was a teaching maths and there was a teaching Telugu, which is a local language.

295
00:33:32,210 --> 00:33:38,560
They're also much better at teaching Hindi because they do teach Hindi which public school don't do.

296
00:33:38,560 --> 00:33:45,970
So what did we learn here? Well, it's not that private schools have a different technology that somehow works better.

297
00:33:45,970 --> 00:33:49,490
They are pushing there, but they are putting that effort differently.

298
00:33:49,490 --> 00:33:55,090
They are putting that effort on English instead of putting it in Telugu and maths,

299
00:33:55,090 --> 00:33:58,680
which lead to actually what performance there are with better performance in English.

300
00:33:58,680 --> 00:34:09,890
Presumably the response to Japan demand. So what Fundament is not really a basic skill of local language, and maths with on demand is English.

301
00:34:09,890 --> 00:34:19,280
And that leads to a possible explanation of of of what the key issue is if kids can learn about it.

302
00:34:19,280 --> 00:34:26,740
What could the problem be? And this is what we have been calling the tyranny of the curriculum.

303
00:34:26,740 --> 00:34:32,260
The problem is not the incentives of doing our job as teacher perceives to be the job.

304
00:34:32,260 --> 00:34:37,450
The problem is the perception of teach us to be a joke, a product of what is done to them.

305
00:34:37,450 --> 00:34:42,700
The problem is that teachers do not think that their job is to teach kids to read or to hide.

306
00:34:42,700 --> 00:34:51,790
The problem is that teachers think that their job is to complete the curriculum and they are not wrong because our job is to complete the curriculum.

307
00:34:51,790 --> 00:35:02,920
In fact, if you are looking at the result, there is a small I didn't show you these classes initially with the performance from 2005 to 2010.

308
00:35:02,920 --> 00:35:06,550
But what you find is actually the test scores going down a little bit.

309
00:35:06,550 --> 00:35:13,240
The reading test scores going down in 2010, and that could be we don't know for a fact,

310
00:35:13,240 --> 00:35:20,530
but that's consistent with a pal west side effect of the Right to Education Act,

311
00:35:20,530 --> 00:35:26,710
which was best to India on that time, which said that every key to the right to education on Nice in the world,

312
00:35:26,710 --> 00:35:32,440
but also said the teachers are compelled by law to complete the curriculum.

313
00:35:32,440 --> 00:35:35,170
So in a sense, criminalised not doing the curriculum.

314
00:35:35,170 --> 00:35:41,020
It didn't said that teachers are complete by law to teach the kids to to every kid to do we don't like.

315
00:35:41,020 --> 00:35:48,700
So the the emphasis is on the curriculum and the curriculum is is is extremely ambitious.

316
00:35:48,700 --> 00:35:58,600
So I put a pulled up just one little example of what Grade four kids are supposed to be doing in Haryana, which is the state we are.

317
00:35:58,600 --> 00:36:03,220
Delhi is located in maths. Oops.

318
00:36:03,220 --> 00:36:09,740
So they need to in October, all they learn the guts and wheels and then halves and quarters in November.

319
00:36:09,740 --> 00:36:19,530
All they play with batons and stuff to do a table up to 15, who he knows they are the table of multiplication of 14.

320
00:36:19,530 --> 00:36:23,860
I'm not going to do a test because then I would have to do the result of my cell phone.

321
00:36:23,860 --> 00:36:32,860
And then they, once they are done this stuff to understand volumes and then there and then they need to understand perimeters and areas of shapes.

322
00:36:32,860 --> 00:36:43,580
We have talking about grade four. So this is just like. By and in in the sky, but in a very, very faraway sky.

323
00:36:43,580 --> 00:36:50,780
But that's that's what they do. So that's a problem, which is not the India specific.

324
00:36:50,780 --> 00:36:56,030
That's a problem which you find in many countries in particular.

325
00:36:56,030 --> 00:37:01,880
In France, we have kind of the same issue, but mostly also in former colonies,

326
00:37:01,880 --> 00:37:08,450
partly because the education system in the sense was inherited cities from developing

327
00:37:08,450 --> 00:37:15,170
from colonial powers who had created it to create a small elite of educated clerks.

328
00:37:15,170 --> 00:37:20,420
And then when the system was passed on at independence, it wasn't.

329
00:37:20,420 --> 00:37:27,120
I completely see. I completely get that it wasn't feasible to dumb down at the moment where you

330
00:37:27,120 --> 00:37:34,280
were going to expand it politically or kind of not a very visible gesture.

331
00:37:34,280 --> 00:37:41,300
But the researchers developed a mindset and a curriculum that's just completely inappropriate,

332
00:37:41,300 --> 00:37:47,300
not just for these kids, but for any kid and any and any teachers.

333
00:37:47,300 --> 00:37:51,740
And if you take Finland, for example, which is doing, you know,

334
00:37:51,740 --> 00:37:58,100
acing the test scores across all of the international test, the curriculum is not at all ambitious.

335
00:37:58,100 --> 00:38:05,510
In fact, kids do not start to learn to to to start, to learn, to read before they they they are seven.

336
00:38:05,510 --> 00:38:12,290
So mind you, that's not a causal, you know, that's not a I'm not trying to infer too much from that,

337
00:38:12,290 --> 00:38:17,090
but that's just to say that it's not like an Indian peculiarity.

338
00:38:17,090 --> 00:38:23,510
Overambitious curriculum is is a plague, and it's something that is very, very difficult to change.

339
00:38:23,510 --> 00:38:27,420
And what? The political conversation is very hard.

340
00:38:27,420 --> 00:38:31,840
So what do you do then? Well, back to them.

341
00:38:31,840 --> 00:38:34,650
So time is this organisation that does so.

342
00:38:34,650 --> 00:38:44,280
But even before doing so, they started working on education early on in the mid 90s when they got stuck to their motto was every child in school.

343
00:38:44,280 --> 00:38:51,960
And very quickly they added every child in school and learning. And finally, they added every child in school and learning well.

344
00:38:51,960 --> 00:38:57,450
And now if you talk to a mother who is the former chairman of POTEM, they want to learn in school.

345
00:38:57,450 --> 00:39:05,190
Is it just every child learning was because the school board doesn't seem to be all that Hollywood to him.

346
00:39:05,190 --> 00:39:08,940
But so what is the what's the approach of them?

347
00:39:08,940 --> 00:39:17,130
It's very simple when you think about it, given the diagnostic I just put is that you need to teach kids where they are able to to learn.

348
00:39:17,130 --> 00:39:21,510
So you need to recoup the the kids.

349
00:39:21,510 --> 00:39:27,320
You need to assess the kids vision, which is a very simple tool for focussing on basic skills.

350
00:39:27,320 --> 00:39:32,460
Then you need to avenge them by what they know at moment.

351
00:39:32,460 --> 00:39:37,830
And then they have a set of activities that can be done either by a teacher or volunteer to run that.

352
00:39:37,830 --> 00:39:45,060
And then you regroup frequently because once the kid is in the right level, you could actually progress fast to the levels.

353
00:39:45,060 --> 00:39:49,710
So that's the top of the core of their approach that it can be modulated in any sort of way.

354
00:39:49,710 --> 00:39:56,570
You can do it with computers, you can do it with people. Teachers can do it with volunteer.

355
00:39:56,570 --> 00:40:08,090
So I'm not going to spend too much time on that, partly because I have only five minutes left, but it's the evaluation of the towel,

356
00:40:08,090 --> 00:40:14,870
which was not yet co-written with Balzac at the time, was my first condom is the control trial that I was involved with.

357
00:40:14,870 --> 00:40:25,230
And we've been working on that for about on and off ever since we started by demonstrating to under my control trial quote.

358
00:40:25,230 --> 00:40:30,530
Tom's approach works very well when it's done with a volunteer.

359
00:40:30,530 --> 00:40:40,880
We started the first people show that was about was in urban areas, then with which we worked in the rural areas, in the with the.

360
00:40:40,880 --> 00:40:46,130
And then so it seems to be that the result replicated for context, to context, very similar.

361
00:40:46,130 --> 00:40:50,730
We are from other countries as well. So that seems to be like.

362
00:40:50,730 --> 00:40:59,310
An approach that not only makes sense in theory, but happens to work in practise and believe me, that's not true for everywhere you plug.

363
00:40:59,310 --> 00:41:07,260
So now an issue that the problem the programme wanted to know is that the take up of the programme is quite low because remember parents,

364
00:41:07,260 --> 00:41:15,240
they don't they don't really see the learning, the key basic skills as something which is fundamentally important.

365
00:41:15,240 --> 00:41:23,760
So Padam struggled with attracting kids to their camps, even though the camps was very effective for kids who did come.

366
00:41:23,760 --> 00:41:27,600
About eight percent of the kids did come in the summer, for example, to attend the camp,

367
00:41:27,600 --> 00:41:31,800
so the effect on the overall population was lower than it could have been.

368
00:41:31,800 --> 00:41:36,180
And that really led them to think that they have to work in schools.

369
00:41:36,180 --> 00:41:43,500
So the problem when you work in schools is that you have to convince the higher authorities that they have to understand that there is a problem,

370
00:41:43,500 --> 00:41:52,020
which is not to which is not always easy and different Indian states and then but that they have been successful at doing that in seven states.

371
00:41:52,020 --> 00:41:58,290
That's problem. One problem, too, is you have to convince teachers to actually take it on board.

372
00:41:58,290 --> 00:42:05,760
The problem is the programme has to take in the organisation and that's what I've called a class of plumbing issue,

373
00:42:05,760 --> 00:42:19,170
which is that it is not a it is not deep philosophical problem, but it is very hard to get something new to come in and a new organisation.

374
00:42:19,170 --> 00:42:28,020
And to cut a long story short, it took us many, many different trials to find a version of the plug on two versions of the programme.

375
00:42:28,020 --> 00:42:32,070
Actually, that does work in primary school,

376
00:42:32,070 --> 00:42:37,890
who first tried and we failed and we understood what we failed and we did it again and then tried again and again.

377
00:42:37,890 --> 00:42:44,640
And finally, we have something that works. In fact, we have two two models that does work in one model.

378
00:42:44,640 --> 00:42:50,150
It is volunteer to do it, but they do it in schools, so they basically take over all the school.

379
00:42:50,150 --> 00:42:59,780
For 50 days, that works hidden in places where the education system is very, very broken, so teachers don't really mind getting 50 days of vacation.

380
00:42:59,780 --> 00:43:07,160
But how many cases where actually they wouldn't mind because they feel that they are serious people, they feel they have a job to do so.

381
00:43:07,160 --> 00:43:14,390
Another veteran of the programme, which is easier to scale up, actually, is to work two teachers trained to teach us to implement it.

382
00:43:14,390 --> 00:43:19,130
But then you really have to signal very strongly that it's their job to do that.

383
00:43:19,130 --> 00:43:25,730
That requires to train also the teacher supervisor and that required to isolate a specific moment in the day,

384
00:43:25,730 --> 00:43:32,240
which can only be used for these activities. And so we decided to on an experiment.

385
00:43:32,240 --> 00:43:42,590
We've found good results from them. This is the how that is the teacher led model where you can see the the yellow ones saw the difference at N-Line.

386
00:43:42,590 --> 00:43:46,820
Between treatment and control, you can see the treatment is higher than control.

387
00:43:46,820 --> 00:43:51,050
Uttar Pradesh is a volunteer model. The effects are much larger.

388
00:43:51,050 --> 00:43:58,610
So when you can do it, that's nice. You can see that they start from an extremely low level, but then they end up mostly almost that.

389
00:43:58,610 --> 00:44:05,900
They have end level at N-Line, starting from a much lower place in the dark blue bar over there.

390
00:44:05,900 --> 00:44:14,340
So this is the volunteer led model in schools, which was done in this state where the education system is pretty much defunct.

391
00:44:14,340 --> 00:44:19,860
So the advantage of doing that is now we have kind of we have cut them has in their hand

392
00:44:19,860 --> 00:44:26,130
a model of two models that can be scaled up and they they have been scaling it up.

393
00:44:26,130 --> 00:44:36,720
The India scale up as of September 2017 was sucked stage twenty 28000 scores of four million kids.

394
00:44:36,720 --> 00:44:42,780
The dialect learning model is much fewer schools, but continues to play a role.

395
00:44:42,780 --> 00:44:50,820
There is no randomised evaluation of that, just the proposed data, which seems to suggest good adoption and some, some some progress.

396
00:44:50,820 --> 00:44:59,850
What I think is might be a little bit of a start itself is a transition to twitchers intervention.

397
00:44:59,850 --> 00:45:08,460
Is that now we got the chance with funding from a large new called in fact to start

398
00:45:08,460 --> 00:45:14,730
responding to a demand for the expansion of this programme in various other countries,

399
00:45:14,730 --> 00:45:23,230
in particular in several countries in Africa. So you can see here all of the countries that have expressed interest are where a pilot has started.

400
00:45:23,230 --> 00:45:34,380
The nice thing is that it's heavily supported by Khartoum, so the African team go to India and the India team go to Africa at some point.

401
00:45:34,380 --> 00:45:40,200
My sister was involved a long time ago in it and sister of that my sister was involved in.

402
00:45:40,200 --> 00:45:50,640
It says that I need to stop talking. My my sister was involved in that and she lost the Kenyan director of education in the middle of India.

403
00:45:50,640 --> 00:46:00,510
So it was it created a little diplomatic crisis, but that that got solved and the UN can now move on.

404
00:46:00,510 --> 00:46:09,570
So if you left for me one minute, I'll conclude with some thoughts on where do we go from here to fix primary education?

405
00:46:09,570 --> 00:46:14,790
So one option is to give up on school, said the school double. And this is a little bit, you know,

406
00:46:14,790 --> 00:46:21,750
maybe you might have second inclination when I was saying they if he wants to drop in school from every child learning well,

407
00:46:21,750 --> 00:46:28,980
there are a lot of digital efforts that are taking place mostly out of school by distributing tablets to kids.

408
00:46:28,980 --> 00:46:33,860
So it has a great potential, extremely large impact of some initiatives to, for example,

409
00:46:33,860 --> 00:46:39,210
mind Spark, which is the machine learning based software programme to learn.

410
00:46:39,210 --> 00:46:44,970
But the fundamental problem with this approach is that kids are in school.

411
00:46:44,970 --> 00:46:50,970
So there you have them, hostage for whatever you want. Everything else require them to show up.

412
00:46:50,970 --> 00:46:55,560
And because the demand for basic skills is not there, they don't show up.

413
00:46:55,560 --> 00:47:01,110
For example, in the US, after the very successful evaluation of mine Spark software,

414
00:47:01,110 --> 00:47:06,300
which was done after school, the parents willingness to continue with the programme was zero.

415
00:47:06,300 --> 00:47:13,560
They were just not interested because while it helped the kids reach basic skills, it didn't help them go to grade level.

416
00:47:13,560 --> 00:47:18,330
So the parents were like, Why do we care? It's not going to get them in it. And in fact, it is not.

417
00:47:18,330 --> 00:47:24,210
But it might still have. Some have some use. So I don't think we can abandon the schools.

418
00:47:24,210 --> 00:47:33,660
Changing the curriculum, of course, is the is the holy grail. The problem is that the political fight is a pin.

419
00:47:33,660 --> 00:47:40,470
I do think that there is a glimmer of hope in India. In Delhi, they have just decided to give away the officially said.

420
00:47:40,470 --> 00:47:46,260
We are not going to finish the curriculum. And more perhaps even more striking,

421
00:47:46,260 --> 00:47:52,470
the central government also the minister indicated the desire to simplify the curriculum very recently

422
00:47:52,470 --> 00:47:58,590
a couple of months ago and announced a commission to to think about it now with the commission.

423
00:47:58,590 --> 00:48:02,400
Who knows? You can also work on the margins.

424
00:48:02,400 --> 00:48:10,530
There are some parts of the system that are more open a pre-school where we started working on the pre-school mathematics,

425
00:48:10,530 --> 00:48:17,760
early grades, even in grade one. There is less ambition to do the summer tutoring.

426
00:48:17,760 --> 00:48:21,750
Different schools. We can use the margin and we've been doing that.

427
00:48:21,750 --> 00:48:32,970
The. But finally, I think ultimately there is not going to be one thing that is going to solve the problem very, very fast.

428
00:48:32,970 --> 00:48:37,440
It's one way I mean, I would love to be quick, but it's not going to happen.

429
00:48:37,440 --> 00:48:42,690
Ultimately, you need to continue to engage with the system and one by one.

430
00:48:42,690 --> 00:48:51,360
And however, splitting the parties get them to adopt things like that help kids coping with the curriculum.

431
00:48:51,360 --> 00:48:55,520
So that helps teachers ignore the curriculum or things like that.

432
00:48:55,520 --> 00:49:02,460
So the challenge is to exploit the existing wedges and to pry open new ones at every opportunity.

433
00:49:02,460 --> 00:49:07,180
This is not going. So this doesn't create like I have the solution you just adopted.

434
00:49:07,180 --> 00:49:11,640
But those are still the gains that we are getting in the meantime.

435
00:49:11,640 --> 00:49:18,780
And of course, any real gains you get is quickly multiplied by millions and millions of children if you're working at the level of a school system.

436
00:49:18,780 --> 00:49:27,780
So that's kind of the the the sort of not maybe earth shattering evolutionary way that we are going at it,

437
00:49:27,780 --> 00:49:34,050
but which has, I think, some real progress to show for itself.

438
00:49:34,050 --> 00:49:50,310
For example, the median of kids that I just mentioned, Peter, thank you very much.

439
00:49:50,310 --> 00:50:07,820
Thank you very much. And now Rachel Bannister is going to give us a different perspective on the education problem.

440
00:50:07,820 --> 00:50:14,930
Thanks. So it's great to be here. It's great to be back with Esta sharing the stage again.

441
00:50:14,930 --> 00:50:21,680
It's great to be a stone's throw from my old colleagues, Somerville right next door.

442
00:50:21,680 --> 00:50:30,440
So I'm going to follow up a bit on on what Esther said and talk about three issues.

443
00:50:30,440 --> 00:50:38,360
First, look at some costing estimates, so you've heard a lot about kind of what's effective in improving education.

444
00:50:38,360 --> 00:50:44,180
But you know what? What are some of the costs associated with some of these interventions?

445
00:50:44,180 --> 00:50:50,660
And some of that is going to tell some similar stories, but maybe from a slightly different perspective.

446
00:50:50,660 --> 00:50:57,230
And then I want to delve into the question of how do you design improvements in individual countries?

447
00:50:57,230 --> 00:51:05,270
So we've heard a lot about this overarching evidence about what's effective and what's wrong with a deep dive into India.

448
00:51:05,270 --> 00:51:12,950
But you know what, if you're going into Somalia, say, which is, you know, I was working with the different team in Somalia recently.

449
00:51:12,950 --> 00:51:13,670
And you know,

450
00:51:13,670 --> 00:51:22,700
how do you think about the problems there and how do you use evidence from other places around the world to inform what you're going to do that?

451
00:51:22,700 --> 00:51:28,130
And then I'll talk just a little bit about what different is doing in this area and

452
00:51:28,130 --> 00:51:33,530
in trying to shape the incentives globally to think about some of these issues.

453
00:51:33,530 --> 00:51:44,600
So before I start talking about costs, let me just explain a little bit about how we try and measure cost effectiveness in education.

454
00:51:44,600 --> 00:51:47,270
And there's no really great way of doing it.

455
00:51:47,270 --> 00:51:57,020
But the kind of the standard in the literature is to look at the standard deviation improvements in learning per $100 spent.

456
00:51:57,020 --> 00:52:03,500
And that's not great. We would love to have some absolute standard of learning, but you know,

457
00:52:03,500 --> 00:52:12,650
if you're trying to compare learning and a study done in India to one done in Kenya or one in the US, everyone's at a different learning level.

458
00:52:12,650 --> 00:52:16,250
And how do you compare an improvement in, you know,

459
00:52:16,250 --> 00:52:26,210
getting a kid to be able to read a sentence versus someone who's getting a kid to add single digits to single digit addition?

460
00:52:26,210 --> 00:52:31,880
So so what we fall back on is looking at standard deviation improvements.

461
00:52:31,880 --> 00:52:39,240
And, you know, so I guess I'm in the mathematical institute, so maybe I shouldn't have to explain all the standard deviation is, but,

462
00:52:39,240 --> 00:52:49,490
you know, point two of a standard deviation moving from the 50th of the 50th percentile one in education terms, one standard deviation.

463
00:52:49,490 --> 00:52:55,400
That's a lot of learning. That's a decent amount of learning in a year is one way to measure it.

464
00:52:55,400 --> 00:53:02,510
So, you know, if you spend hundred dollars and you get a year's worth of learning and a decent system that's like phenomenally cost effective.

465
00:53:02,510 --> 00:53:11,210
When I show these results to people in the US or the UK, they think I've got the decimal point wrong because it's just, you know,

466
00:53:11,210 --> 00:53:19,040
blown away by how much learning you can get from some of these effective interventions for relatively little cost,

467
00:53:19,040 --> 00:53:25,270
which is actually true in lots of areas of development.

468
00:53:25,270 --> 00:53:31,420
So the first thing I want to say, and maybe it's a little bit of a tweak on what Esther was saying,

469
00:53:31,420 --> 00:53:35,830
which is about access because it's completely true.

470
00:53:35,830 --> 00:53:47,350
So are you getting kids into school? It's completely true that kids are not learning anything like as much as they should be in school.

471
00:53:47,350 --> 00:53:52,300
And I think India is particularly depressing because you're seeing learning go down.

472
00:53:52,300 --> 00:54:01,440
But where? But I don't think we should give up on the fact, you know, we made huge progress in getting kids in school in,

473
00:54:01,440 --> 00:54:08,320
you know, in sub-Saharan Africa in particular over the last 10 and 20 years.

474
00:54:08,320 --> 00:54:14,170
I don't think that was a complete waste of effort and some of it.

475
00:54:14,170 --> 00:54:26,230
When you look across all the randomised trials, you see you see some trials that had big increases in access did lead to increases in learning.

476
00:54:26,230 --> 00:54:30,790
Now it was increasing learning on school maths, you know,

477
00:54:30,790 --> 00:54:38,990
in-school learning so that that we should take what Esther was saying about that into account when we think about that.

478
00:54:38,990 --> 00:54:45,250
But it is the case that if you go into Afghanistan and create schools where there were no schools,

479
00:54:45,250 --> 00:54:53,980
kids, no more in the schools where those schools were created. So, so you know, that's and it's also interesting.

480
00:54:53,980 --> 00:55:03,070
There was a lot of work done on conditional cash transfers, which improved the number of kids going to school in Latin America and a lot of

481
00:55:03,070 --> 00:55:10,810
depression about the fact that they saw no increase in in test scores in those schools.

482
00:55:10,810 --> 00:55:17,680
Well, it turns out that if you look carefully at those studies because you were moving kids from, you know,

483
00:55:17,680 --> 00:55:22,600
maybe 80 percent of kids going to school to eighty five percent,

484
00:55:22,600 --> 00:55:28,240
you simply didn't have the statistical power to have picked up a reasonable increase in learning scores.

485
00:55:28,240 --> 00:55:35,270
So I'm slightly more. You know, we we shouldn't assume that access is, you know, not at least a first step.

486
00:55:35,270 --> 00:55:43,630
That doesn't mean to say that, you know, we got a lot to do in terms of improving learning within school.

487
00:55:43,630 --> 00:55:47,260
The next thing that comes across from this, you know, cost effectiveness.

488
00:55:47,260 --> 00:55:53,590
So all of the balls in these charts are results from a randomised trial.

489
00:55:53,590 --> 00:56:02,590
And so this is for which we have cost information as well of as well as impact information.

490
00:56:02,590 --> 00:56:10,600
And you know, the things to take away from this slide is to reinforce the point that simply more inputs is just not getting,

491
00:56:10,600 --> 00:56:12,430
you know, if you teach the same way,

492
00:56:12,430 --> 00:56:22,510
but you have more textbooks or you have, you know, computers, you it doesn't do anything to learning because the fundamental issue is how we teach.

493
00:56:22,510 --> 00:56:28,870
Well, what's interesting is one. So one of the charts that shows an effect is school grants.

494
00:56:28,870 --> 00:56:32,680
And then you look at the second year of that same programme and it goes back to zero.

495
00:56:32,680 --> 00:56:41,620
The only programme that has an effect is giving textbooks when you look just at the top of the class.

496
00:56:41,620 --> 00:56:47,380
So the kids who are already performing well, they're the textbooks actually did have an effect.

497
00:56:47,380 --> 00:56:57,520
And it's actually a reasonably cost effective intervention because you get over a standard deviation increase in learning for those kids.

498
00:56:57,520 --> 00:57:04,670
Why? Because they're the only kids who understand the textbook, so which reinforces the point that the curriculum is completely wrong.

499
00:57:04,670 --> 00:57:12,290
So it's working for a very small slice of kids, but not for all of them.

500
00:57:12,290 --> 00:57:16,520
Now, this is something that really surprised me when we did this cost effectiveness was

501
00:57:16,520 --> 00:57:23,060
because there's a lot of work to try and improve the way schools are managed.

502
00:57:23,060 --> 00:57:32,870
And many of the studies don't have an impact. But when I started looking at the cost per impact, when you when it does work,

503
00:57:32,870 --> 00:57:39,080
it's phenomenally cost effective because it doesn't actually cost much to improve the way that schools are managed.

504
00:57:39,080 --> 00:57:49,970
And that's about feeding back information into the system, about about whether teachers are showing up and whether the system is working.

505
00:57:49,970 --> 00:57:56,840
Now this is you know what Esther talked a lot about, which is how do you changes in how teachers teach?

506
00:57:56,840 --> 00:58:02,450
And that's consistently very effective. But just look at some of those results.

507
00:58:02,450 --> 00:58:11,270
I mean, we're talking about, you know, 30 full standard deviation improvements for $100 dollars.

508
00:58:11,270 --> 00:58:16,310
I mean, that's just phenomenally cost effective because you've got the kids,

509
00:58:16,310 --> 00:58:23,360
you've got the schools, you've got the teachers, and all you're doing is changing how they teach.

510
00:58:23,360 --> 00:58:30,800
So there's a fantastic opportunity to take the systems that we've built over the last 20 years and improve,

511
00:58:30,800 --> 00:58:38,390
improve how we teach in the systems at relatively little additional cost.

512
00:58:38,390 --> 00:58:41,300
Esther talked a bit about accountability.

513
00:58:41,300 --> 00:58:48,590
You know, accountability is matters, but it's really hard to get right, I think, as my summary of this literature.

514
00:58:48,590 --> 00:58:56,120
Let me go on to talk a bit about how we take this evidence to two individual contexts,

515
00:58:56,120 --> 00:58:59,630
because a lot of questions get asked when we talk about evidence about.

516
00:58:59,630 --> 00:59:04,400
But isn't it different in different contexts and how do you take the context into account?

517
00:59:04,400 --> 00:59:11,480
Well, you know, we've had this massive increase in the number of impact evaluations in it in education,

518
00:59:11,480 --> 00:59:19,400
but we still think that, you know, context matters. So how do you square that circle because you're never going to have an impact evaluation

519
00:59:19,400 --> 00:59:24,770
exactly the question that you care about in exactly the context where you're working?

520
00:59:24,770 --> 00:59:31,830
Well, this is my, you know, framework for thinking about how you do it.

521
00:59:31,830 --> 00:59:35,790
You know, you have to start by diagnosing what is the local failure,

522
00:59:35,790 --> 00:59:41,130
what's the local challenge in a given context, and that's going to be different in different places.

523
00:59:41,130 --> 00:59:48,900
And then you want to draw on general lessons of behaviour about, you know, how kids learn,

524
00:59:48,900 --> 00:59:53,730
which is us to talk about is actually quite similar in many ways across contexts.

525
00:59:53,730 --> 00:59:59,650
And then you have to worry about the plumbing. How do you make it work in this context?

526
00:59:59,650 --> 01:00:07,410
Right? And that may be very different. It may be very different in a place where you have computers already and people who know how to use

527
01:00:07,410 --> 01:00:13,830
them versus ones that what you don't or where you have volunteers who could work in the system,

528
01:00:13,830 --> 01:00:20,790
et cetera. So this is how I would take this talk and kind of put it into that format.

529
01:00:20,790 --> 01:00:29,730
So this teaching at the white level? If the diagnosis of the problem is, as it often is, children are in school or not learning,

530
01:00:29,730 --> 01:00:34,140
there's high variation in learning levels within the class.

531
01:00:34,140 --> 01:00:41,940
Teachers are focussed on teaching the curricula and not on learning, and many children are below the curricula.

532
01:00:41,940 --> 01:00:49,890
Then you can draw on this body of evidence, and you heard a lot about the Pratham version of teaching at the right level and a little bit at the end.

533
01:00:49,890 --> 01:00:58,830
But let me just explore a bit on that. There are other approaches to trying to tailor the level of teaching to where the children are,

534
01:00:58,830 --> 01:01:03,900
which is what is not, you know, which has it, which is also positive.

535
01:01:03,900 --> 01:01:09,360
So, you know, using where you have, you know, lots of computers in school,

536
01:01:09,360 --> 01:01:18,650
using adaptive computer software can be very effective way to adapt to the level of the child.

537
01:01:18,650 --> 01:01:25,310
Right. Tracking by just taking the existing system and saying,

538
01:01:25,310 --> 01:01:31,730
let's put thought children by where they are at the beginning of the year was also very effective

539
01:01:31,730 --> 01:01:36,830
way of teaching at the right level and didn't actually involve changing the curriculum.

540
01:01:36,830 --> 01:01:40,580
That wasn't, as you know, I think you can probably do better.

541
01:01:40,580 --> 01:01:46,400
That's at least a step of teaching at the right level. So there's many different approaches.

542
01:01:46,400 --> 01:01:52,640
And when we go in and talk to a country about how they can use this learning and adapt it to their environment,

543
01:01:52,640 --> 01:01:58,760
it's always useful to go in with many different ideas about how they could bring that off.

544
01:01:58,760 --> 01:02:04,340
And then you have to think about local implementation. What can you pull off in this environment?

545
01:02:04,340 --> 01:02:06,800
Is it remedial education after school?

546
01:02:06,800 --> 01:02:16,850
Is it personalised learning software or is it tracking, which is the intervention that makes sense in the local context or changing the curriculum?

547
01:02:16,850 --> 01:02:22,550
I'm running out of time, but let me just say, you know, the diagnosis is not always the same in every place.

548
01:02:22,550 --> 01:02:30,410
So when my conversations in Somalia, you know, 17 percent of primary age children are in school.

549
01:02:30,410 --> 01:02:40,530
So there we have to start with the first challenge of how do we provide schools in a conflict affected area where the state isn't really working?

550
01:02:40,530 --> 01:02:47,060
Right. So can we take the evidence from Afghanistan and think about community based schools?

551
01:02:47,060 --> 01:02:53,690
What we do? And but we can still learn from the general evidence about how to get kids in school,

552
01:02:53,690 --> 01:03:00,650
which is we know that convenient, cheap access is a really important way to get so.

553
01:03:00,650 --> 01:03:08,450
So if you make schools free and you make them convenient, actually most kids will go.

554
01:03:08,450 --> 01:03:15,950
So it's very sensitive to price and convenience and distance is really important, especially for girls.

555
01:03:15,950 --> 01:03:21,620
You know, again, local implementation, you've got to think about what you can pull off in a conflict affected area.

556
01:03:21,620 --> 01:03:29,680
Building permanent structures, maybe not what you want to do. You may want to use local community groups.

557
01:03:29,680 --> 01:03:37,300
So briefly, what is what is the Fed doing? One of the things that we're thinking about is how to change the incentives in the system.

558
01:03:37,300 --> 01:03:42,280
So we, you know, we talked about how the curriculum is overly ambitious.

559
01:03:42,280 --> 01:03:46,750
How do we get people to focus on basic skills?

560
01:03:46,750 --> 01:03:56,650
Right. So one way is to do something like Pressman has been doing and measure basic skills and publish data on basic

561
01:03:56,650 --> 01:04:03,460
skills to try and get education systems to focus on basic reading and writing rather than at the moment.

562
01:04:03,460 --> 01:04:08,530
They will focus on getting kids through that final high stakes exam at the end of secondary school,

563
01:04:08,530 --> 01:04:11,410
which I knew a few people are going to get through.

564
01:04:11,410 --> 01:04:20,800
So I don't know if you heard recently about Jim Kim's initiative to produce a human capital index, but that's, you know, one of the Bennett.

565
01:04:20,800 --> 01:04:28,090
One of the ideas behind that is to have kind of an index of what, what people are learning, what kids are learning.

566
01:04:28,090 --> 01:04:36,070
And very important to that will be measuring whether people are acquiring these basic literacy and numeracy skills.

567
01:04:36,070 --> 01:04:40,330
The other thing we're doing is investing in learning about learning. What are we?

568
01:04:40,330 --> 01:04:50,710
What do we know about how to improve learning in school? So some of the work that that is just talked about has been has been funded in this way.

569
01:04:50,710 --> 01:04:56,440
And finally, thinking about how do we take some of these results to scale?

570
01:04:56,440 --> 01:05:01,510
And I want to just briefly talk about an experiment in and scaling up in Kenya.

571
01:05:01,510 --> 01:05:11,080
We did try and tackle the curricular issue. So it said, you know, working with the Kenyan government, let's test taking the curricula,

572
01:05:11,080 --> 01:05:20,860
making it more appropriate to what kids know and to the lives, you know, in the education system and where the kids are at and test that small scale.

573
01:05:20,860 --> 01:05:27,160
And it was very effective. And then what were the Kenyan government to change the curricula across all of the

574
01:05:27,160 --> 01:05:34,030
early years and is now being scaled up to twenty three thousand primary school?

575
01:05:34,030 --> 01:05:38,080
So you can only work on the curriculum where people are open to it,

576
01:05:38,080 --> 01:05:45,820
but it's absolutely fundamental to changing the the incentives and getting children learning across the world.

577
01:05:45,820 --> 01:06:03,660
Thank you. Thank you very much, Rachel and Richard Peto is going to be our final speaker.

578
01:06:03,660 --> 01:06:08,810
OK. I would take the slides off, please.

579
01:06:08,810 --> 01:06:16,750
It's hard to take that one off as well. I can't stand looking at my face. OK.

580
01:06:16,750 --> 01:06:24,690
Yeah. Well. One thing I came to Oxford in about the 90s in the nineteen sixties, as well as same decade as Sanjay,

581
01:06:24,690 --> 01:06:35,400
and I've mostly been doing medical research since then, and I wanted to try to think about some of the analogies and most of it's been on trial.

582
01:06:35,400 --> 01:06:41,710
When I first started working on randomised trials in medicine back in 1967 and in some.

583
01:06:41,710 --> 01:06:50,320
I think some of the lessons of randomised trials in medicine will carry over into the social sciences,

584
01:06:50,320 --> 01:06:53,530
you know, randomising, was it due to the criminal impact?

585
01:06:53,530 --> 01:07:00,790
You know, if you have this sentence or double their sentence and randomising, what does it do in terms of education?

586
01:07:00,790 --> 01:07:06,340
And I found this book, the poor economics book, really citing, I read it.

587
01:07:06,340 --> 01:07:12,010
I thought I'd read it about two or three years ago, and in preparation of this panel,

588
01:07:12,010 --> 01:07:15,070
I reread it and I'll tell you, I find it just much, much better on rereading.

589
01:07:15,070 --> 01:07:19,810
That's partly because knowing I'm going to be sitting standing next to concentrated my mind a bit,

590
01:07:19,810 --> 01:07:26,640
but it's it's really good to read about taking classes and just.

591
01:07:26,640 --> 01:07:31,330
Taking the class as the unit of randomisation and then taking off, say,

592
01:07:31,330 --> 01:07:39,300
the bottom 40 percent and getting them just to a really cheap teaching assistant who gets paid about a tenth as much as the teacher,

593
01:07:39,300 --> 01:07:46,570
try and actually teach them something that's at their level and then put them back in the class and see what the class is as a whole.

594
01:07:46,570 --> 01:07:55,500
Do better if you educate, if you try and deal at very little cost with those who are really not doing well in that class,

595
01:07:55,500 --> 01:08:03,900
and the answer was yes, it did. And how'd you know it did? Because they randomised, they randomised and they got results for the class as a whole.

596
01:08:03,900 --> 01:08:09,570
And then they did actually discuss thoughtfully whether it was because actually just having the 40 percent of most

597
01:08:09,570 --> 01:08:15,990
troublesome kids taken out of the way let the other ones learn something and decided it wasn't that that was a non-writing,

598
01:08:15,990 --> 01:08:24,360
randomised analysis. I mean, it's certainly worth thinking about, but the the idea of randomising is is just very attractive.

599
01:08:24,360 --> 01:08:32,490
But I want to go back and think a bit about what went wrong with randomised evidence in medicine and randomised evidence has been brilliant.

600
01:08:32,490 --> 01:08:35,430
It continues to be brilliant, but it did in various respects,

601
01:08:35,430 --> 01:08:42,000
go go wrong and it is continuing to go wrong in various respects and randomised evidence in medicine.

602
01:08:42,000 --> 01:08:45,750
There are few early efforts to get do some randomised comparisons.

603
01:08:45,750 --> 01:08:52,440
The one that really lit the fire that was the spark that actually produced the conflagration produced the massive

604
01:08:52,440 --> 01:09:01,470
amount of randomised evidence was a trial done back in the 1940s of the use of streptomycin to treat tuberculosis.

605
01:09:01,470 --> 01:09:09,090
A TB was a terrible disease think by the 1940s, it wasn't nearly as many people as it had been, presumably a bit better nourished.

606
01:09:09,090 --> 01:09:12,600
But still, it was a major issue back in the 19th century,

607
01:09:12,600 --> 01:09:17,910
and it used to be referred to as the chief of the captains of death, which is a lovely phrase.

608
01:09:17,910 --> 01:09:22,410
The registrar general doesn't use words like that in the registrar general's report now,

609
01:09:22,410 --> 01:09:28,500
but that was what was just how it was described actually by the registrar general's report in the 1860s.

610
01:09:28,500 --> 01:09:38,460
And suddenly they got streptomycin. There was a drug that could actually really attack the Chibuku Bacillus, and they've got limited supply of it.

611
01:09:38,460 --> 01:09:42,180
They allocated randomly. Half the patients with horrible TB didn't get it.

612
01:09:42,180 --> 01:09:46,800
Half did, and there was a serious difference in survival mean the half who did.

613
01:09:46,800 --> 01:09:53,010
And this was great. It was really exciting to have at last, a drug that could do something against TB.

614
01:09:53,010 --> 01:10:01,550
And also, there was this new technique of choosing at random. Whether or not you should, you should actually use a medicine.

615
01:10:01,550 --> 01:10:08,780
And so somehow the but of course, the truth is that streptomycin was so good that with or without a randomised trial,

616
01:10:08,780 --> 01:10:14,780
its merits would've been recognised and probably your thing with three standard deviations

617
01:10:14,780 --> 01:10:22,160
per $100 spent would be so good that we recognised for that randomised assessment.

618
01:10:22,160 --> 01:10:25,130
And anyway, so then it was in the mid-century.

619
01:10:25,130 --> 01:10:30,680
There were lots of antibiotics being discovered, and also there was polio vaccination that was evaluated by a randomised trial.

620
01:10:30,680 --> 01:10:36,650
But again, actually, polio vaccination works so well that you could have picked up the fact that it was working without randomisation.

621
01:10:36,650 --> 01:10:38,810
It's easy with randomisation,

622
01:10:38,810 --> 01:10:46,610
but you could pick it up that these striking things without randomisation and say randomisation was getting famous for producing,

623
01:10:46,610 --> 01:10:52,250
striking, striking cures battery. That's not what it's useful for.

624
01:10:52,250 --> 01:10:58,250
I mean, yeah, I mean, you could believe that things have striking effects like the power of prayer or homeopathy and so on.

625
01:10:58,250 --> 01:11:04,160
But they don't. But it well, actually, there was that.

626
01:11:04,160 --> 01:11:08,360
This is not what I supposed to be talking about. There was actually randomised trial of the power of prayer,

627
01:11:08,360 --> 01:11:15,530
and it's the only trial that ever actually disproved the null hypothesis by completely nonsignificant results.

628
01:11:15,530 --> 01:11:21,470
And it was done by there was a guy who was president of some statistical society back in the 1970s,

629
01:11:21,470 --> 01:11:26,060
and he, you know, when he was young, he could be brought up with a religious maniac as a friend.

630
01:11:26,060 --> 01:11:29,660
And they, you know, their paths diverge as time has gone by.

631
01:11:29,660 --> 01:11:36,170
But anyway, they still stayed friends. And, you know, he was really excited by statistics and trials and so on back in the 1960s.

632
01:11:36,170 --> 01:11:42,200
And you know, then and his friends stood out, you know, you can't really affect things of life or death by medicine.

633
01:11:42,200 --> 01:11:45,650
You know, these are matters of, you know, the spirits, you know, etcetera, etcetera.

634
01:11:45,650 --> 01:11:50,150
And anyway, one thing led to another and this guy decided he was going to with his friend.

635
01:11:50,150 --> 01:11:53,390
They were going to set up this randomised trial of the power of prayer.

636
01:11:53,390 --> 01:12:00,920
And so they were going to get patients coming into the local hospital who seemed to be similarly ill comparable patients who weren't expected to die.

637
01:12:00,920 --> 01:12:09,980
And the that and they'd go in on the same day, their names we give them to the statistician who would, you know, allocate heads or tails out.

638
01:12:09,980 --> 01:12:15,350
And the local congregation would either prayed for free, pray for Fred to get better or Jim to get better.

639
01:12:15,350 --> 01:12:20,360
And so this was and it was really randomly done. Nobody knew who was being prayed for.

640
01:12:20,360 --> 01:12:23,210
And I don't suppose the parents, the patients, the doctors cared very much.

641
01:12:23,210 --> 01:12:27,050
But anyway, this was going to be the randomised trial of the power of prayer.

642
01:12:27,050 --> 01:12:32,810
And, you know, my friend was sort of sitting back rather smugly thinking that he was going to be good.

643
01:12:32,810 --> 01:12:35,420
This is going to be a triumph of science, French and so on.

644
01:12:35,420 --> 01:12:43,400
And the first and if then it was just whoever came out of hospital first, his score either plus one for prayer or minus one for pragmatism.

645
01:12:43,400 --> 01:12:48,950
And so the the first one came out in favour of prayer actually just had happened.

646
01:12:48,950 --> 01:12:54,380
And then sort of the second one and then the third one, rather to the irritation of my friend who was sitting it up.

647
01:12:54,380 --> 01:12:58,850
And then it went on and the end.

648
01:12:58,850 --> 01:13:06,710
Every time every single one came out in favour of the power of prayer and it was really being done, and it got to the point where one more.

649
01:13:06,710 --> 01:13:11,510
And it would have actually hit the boundary and I had to underline the results and declare them significance.

650
01:13:11,510 --> 01:13:17,630
And he said, I find myself praying it's it goes the other way and it did.

651
01:13:17,630 --> 01:13:21,230
And then, you know, from that point on, it went down and went down at six in delegate.

652
01:13:21,230 --> 01:13:27,830
And so this is the only trial. That's but it's a not insignificant result that disproved the null hypothesis.

653
01:13:27,830 --> 01:13:37,910
But more seriously, the trials did get into trouble because they got this tradition of looking for effects that were too big.

654
01:13:37,910 --> 01:13:44,600
And it was when they started that like pretty much your comment that even if you got small gains that apply to millions, actually,

655
01:13:44,600 --> 01:13:48,980
you could have said tens of millions or hundreds of millions of kids because it's not only those that now,

656
01:13:48,980 --> 01:13:55,370
but there's there's going to be there in the 2020s and the 2030s. So you can actually add in medicine.

657
01:13:55,370 --> 01:13:59,630
The key principle is that actually you can save more lives by moderate reduction,

658
01:13:59,630 --> 01:14:07,880
a big cause of death than you can by a big reduction in small cause of death. And so and we have to in medicine.

659
01:14:07,880 --> 01:14:14,990
I mean, after you go to the big spectacular things done. So there's a few more big spectacular things, but mostly you're left with.

660
01:14:14,990 --> 01:14:22,790
Is there a moderate gain or is there no gain? So the role of my role as a trial is over the decades and is I've got to try and distinguish

661
01:14:22,790 --> 01:14:27,830
reliably between the idea that the treatment is completely useless or that it's fairly useless.

662
01:14:27,830 --> 01:14:31,530
Because even if you take effective use history and if you can take 10 percent down to eight percent,

663
01:14:31,530 --> 01:14:36,800
didn't you start applying that to a million people, you know? Well, that's a huge number of lives saved.

664
01:14:36,800 --> 01:14:43,250
And actually, I take that as an example because that's what we got for giving aspirin in the middle of a heart attack.

665
01:14:43,250 --> 01:14:47,090
And but then there are effects that are more moderate than that.

666
01:14:47,090 --> 01:14:53,630
But they apply to millions, tens of millions of patients, you know, maybe hundreds of millions of patients.

667
01:14:53,630 --> 01:14:57,530
If you actually look over the lifetime of the trials of a common disease and you've

668
01:14:57,530 --> 01:15:02,480
really got to get moderate differences and say you've got to get huge trials,

669
01:15:02,480 --> 01:15:09,380
you've got to get huge trials with if it's going to be random, if you randomising classes, then it's going to be huge numbers of classes.

670
01:15:09,380 --> 01:15:12,360
If you randomising patients, it's going to be huge numbers of patients.

671
01:15:12,360 --> 01:15:16,910
If you're looking for mortality, you've got to have huge numbers of deaths and we've got that for certain things,

672
01:15:16,910 --> 01:15:23,600
but we don't have it for enough things. There's lots of really boring questions, but we need to sort out moderate effects.

673
01:15:23,600 --> 01:15:30,830
And I think and it and it trials got into a complete, you know, after the first quarter century,

674
01:15:30,830 --> 01:15:37,250
they were really coming up with completely muddled results and looking at things that ought to work and keep you on saying it was nonsignificant.

675
01:15:37,250 --> 01:15:41,180
Then you were getting some trial. It was saying something didn't work and somebody else said, yes,

676
01:15:41,180 --> 01:15:48,720
it has a huge effect and there's a thing called base there in which this sort of looks too good to be true, then it probably is.

677
01:15:48,720 --> 01:15:55,130
And eventually, the tradition arose of trying to get all the trials that have looked a particular question

678
01:15:55,130 --> 01:15:59,240
and get them all on the same piece of paper and take some sort of weighted average of them.

679
01:15:59,240 --> 01:16:04,220
And you call it a meta analysis or whatever you like, but get all the trials together.

680
01:16:04,220 --> 01:16:13,130
And you know, if you do get striking claims, then again, probably there's some effect there, but it's not as good as you think.

681
01:16:13,130 --> 01:16:20,240
So actually the case of our study in 1980s aspirin acute heart attack, OK, maybe it doesn't actually change 10 percent to eight percent.

682
01:16:20,240 --> 01:16:27,620
That was a five standard deviation difference, but maybe it could change 10 percent to nine percent or at least half as good as the data suggested.

683
01:16:27,620 --> 01:16:33,200
And and it's still widely used now, but also the.

684
01:16:33,200 --> 01:16:40,320
And I think the trials, if they're going to be done, they're really going to have to be done with an emphasis on.

685
01:16:40,320 --> 01:16:48,840
On what are the random errors in the results and a lot of the smarts that were shown didn't actually show that at all in both of your talks.

686
01:16:48,840 --> 01:16:54,960
And if you're talking about cost effectiveness, you know, we know we want to know what's the effect and what's the cost.

687
01:16:54,960 --> 01:17:00,750
And in medicine, usually it's not so difficult to get some idea of what the cost would be.

688
01:17:00,750 --> 01:17:08,130
But, you know, so it's very difficult to measure the effectiveness. But I think, you know, I just wonder what is the what is the cost?

689
01:17:08,130 --> 01:17:15,990
What is the effectiveness of the various things? And what are the uncertainties in these costs and in the effectiveness that needs to be?

690
01:17:15,990 --> 01:17:22,710
It's going to be a much longer, more tedious process than it seems when you first beautiful results.

691
01:17:22,710 --> 01:17:30,720
So medical randomisation in medicine has been much it's sort of settled down into a rather tedious activity,

692
01:17:30,720 --> 01:17:35,610
which, you know, I've been doing all my life. And now and then you get striking results.

693
01:17:35,610 --> 01:17:39,030
And I think that the same thing's going to have to be true of educational interventions.

694
01:17:39,030 --> 01:17:45,420
It's great that randomisation is happening and also these economic randomisation that you were describing in Indonesia.

695
01:17:45,420 --> 01:17:47,700
I mean, it's really, really very, very nice.

696
01:17:47,700 --> 01:17:53,640
And if anybody wants to really get the book, read this and it's better the second time you read it as well.

697
01:17:53,640 --> 01:17:59,730
But it's going to be a really long business before you start getting really

698
01:17:59,730 --> 01:18:06,780
reliable evidence of randomised assessments where you know the idea that it works,

699
01:18:06,780 --> 01:18:10,800
the idea it doesn't work are both plausible, but certainly actually some of the negative things you know,

700
01:18:10,800 --> 01:18:16,560
do these do various incentives work at least a no result from a trial is good

701
01:18:16,560 --> 01:18:20,310
propaganda against the idea that we know it works and therefore we must do it.

702
01:18:20,310 --> 01:18:27,540
I mean, medicine used to be based so much on unjustified certainty, and the trials were good for debunking that,

703
01:18:27,540 --> 01:18:33,430
even though actually they couldn't really prove that there wasn't any effect.

704
01:18:33,430 --> 01:18:35,760
I mean, which adopted a trial of all these.

705
01:18:35,760 --> 01:18:40,500
We know if you've got a peptic ulcer, you have to go to a milk diet and you've got to sort of eat a little bit of food every two hours,

706
01:18:40,500 --> 01:18:42,480
possibly a bit of boiled fish and so on.

707
01:18:42,480 --> 01:18:47,400
And he just randomised them between, you know, give the all these careful, complicated diets to just eat whatever you like.

708
01:18:47,400 --> 01:18:54,450
There was actually no difference in outcome. In retrospect, his trial was much too small to be informative, but it did actually make respectable.

709
01:18:54,450 --> 01:18:59,940
The idea that you know that much was said to be really important could actually be of no importance, whatever.

710
01:18:59,940 --> 01:19:01,500
So they're quite good as propaganda.

711
01:19:01,500 --> 01:19:09,420
But if you want the most serious scientific evidence, then the much longer and more tedious you think the trial did.

712
01:19:09,420 --> 01:19:17,070
In Uttar Pradesh, we randomised a million kids to get vitamin A. every five, every six months or not.

713
01:19:17,070 --> 01:19:20,250
And you know, at the time, the literature said that seven million kids,

714
01:19:20,250 --> 01:19:26,280
but it was randomised by development block, so it was 72 blocks, 36 blocks versus 36 blocks.

715
01:19:26,280 --> 01:19:31,980
It was great. Our study area of seeing for a satellite spent something like three degrees of latitude and one degree of longitude.

716
01:19:31,980 --> 01:19:37,620
But for all that, we we fed the nought point four tonnes of vitamin A. every six months,

717
01:19:37,620 --> 01:19:41,550
and at the end, it was supposed to reduce child mortality by a quarter.

718
01:19:41,550 --> 01:19:46,410
And we finished up the relative risk of nought point nine six for the standard of 0.03.

719
01:19:46,410 --> 01:19:53,070
And you know, at the end, I suspect that if we put all the overoptimistic trials together with our over pessimistic trial,

720
01:19:53,070 --> 01:19:58,440
then probably it would reduce turbo charge by about 10 percent.

721
01:19:58,440 --> 01:20:03,630
But it's, you know, it's been twenty five years of work to get to that point.

722
01:20:03,630 --> 01:20:10,260
And actually, if you do get a trial result like that, we haven't really changed practise because the people who hated vitamin A anyway,

723
01:20:10,260 --> 01:20:14,670
they say we're just we've done a trial that proves it doesn't work and we won't

724
01:20:14,670 --> 01:20:18,420
even accept the end results of the people who were certain it was wonderful,

725
01:20:18,420 --> 01:20:23,910
just as certain that, you know, we must have done an incompetent trial and so they could dismiss it and say somehow or other, it didn't.

726
01:20:23,910 --> 01:20:28,560
Actually, we couldn't even get a consensus on a weighted average of our results and all the other results,

727
01:20:28,560 --> 01:20:32,280
even though ours was twice as big as all the other ones put together. I don't know.

728
01:20:32,280 --> 01:20:37,620
It's it's a it's a bit of a long drawn out, but then you do get some things that are just really beautiful.

729
01:20:37,620 --> 01:20:44,740
You look at the evidence on cholesterol lowering. There's more than 100000 people have been randomised on statins vs. no statins.

730
01:20:44,740 --> 01:20:50,840
But the differences in heart disease mortality, occlusive stroke mortality.

731
01:20:50,840 --> 01:20:58,320
You know, it's beautiful. You just you can just take one little pill and not take any risk of death reclusive vascular disease by about 40 percent.

732
01:20:58,320 --> 01:21:03,330
It would spoil the theatrical effect of that if I dropped it at this point. But still, yeah.

733
01:21:03,330 --> 01:21:07,200
And you know, the side effects also get evaluated by by by.

734
01:21:07,200 --> 01:21:12,090
The randomised trials will be evaluated and the nonsense about statins.

735
01:21:12,090 --> 01:21:18,000
20 years ago, bad cholesterol lowering it was supposed to increase the risk of cancer, increase the risk of suicide and make people more.

736
01:21:18,000 --> 01:21:27,720
And all these things and not to have any net effect. It just simply when you get a drug that does reduce cholesterol that is easy to give and you

737
01:21:27,720 --> 01:21:33,150
actually test it properly with 100000 randomised follow up for about five years with placebo,

738
01:21:33,150 --> 01:21:37,710
then you get you get the right answers. But as I say, this got it turned off.

739
01:21:37,710 --> 01:21:45,560
Take a long time to get from the 1980s when. We will hypothesise that this might work to the present when, yes, it definitely does.

740
01:21:45,560 --> 01:21:51,950
And of course, all we've got now is the Internet Storm about accusations of side effects instead.

741
01:21:51,950 --> 01:21:55,520
And people will not look will not accept the randomised evidence on side effects,

742
01:21:55,520 --> 01:22:01,790
which says that most of what is being said about these massive side effects is nonsense. So I don't know.

743
01:22:01,790 --> 01:22:05,930
It's just I think you've got to you're younger than I am and you've got a longer life ahead.

744
01:22:05,930 --> 01:22:13,700
But it's good. If you are going to really be pushing for randomised evidence, it's going to be a much longer and slower process than you think.

745
01:22:13,700 --> 01:22:34,070
Good luck. Thank you very much for it, it's good to end on a pessimistic note, a decade so optimistic.

746
01:22:34,070 --> 01:22:38,910
If you say it is, the Detroit I'm going to do now got a pretty wonderful results next year, but I'm pessimistic.

747
01:22:38,910 --> 01:22:50,360
But yeah, that's it's OK on average. And thank you to all our speakers for some very stimulating ideas.

748
01:22:50,360 --> 01:23:00,430
We've now time for some questions. So over to you, what would you like to ask them?

749
01:23:00,430 --> 01:23:18,430
Yes, I'm pleased by the military, and it's sort of an opinion piece here in recent days when you can finally get sorted.

750
01:23:18,430 --> 01:23:26,170
It's the kids who are ahead already who tend to benefit most and the kids are behind who don't benefit at all.

751
01:23:26,170 --> 01:23:34,850
I wonder if you to join the gains were made, which children was who were making gains?

752
01:23:34,850 --> 01:23:37,480
That's one of the things he talks about in here.

753
01:23:37,480 --> 01:23:47,350
It posted, So first, what we like to say achievement, not ability, because achievement is what happened.

754
01:23:47,350 --> 01:23:52,090
Whatever happens to be there, what they do now,

755
01:23:52,090 --> 01:23:58,540
which might or might not reflect something fundamental about diabetes, or it's actually not rhetorical.

756
01:23:58,540 --> 01:24:03,700
So when I feel he is right, he is right for academics carefully. We did this trial.

757
01:24:03,700 --> 01:24:06,340
I did exactly this trial in Kenya with Michael.

758
01:24:06,340 --> 01:24:15,040
Come on, Peskin in Dubai, where we, we we randomised some classes in to be touched by achievement or not, for actually two years.

759
01:24:15,040 --> 01:24:22,630
The kids were tracked for two years. So very different than the pattern model, which we re-organizing just treated.

760
01:24:22,630 --> 01:24:31,900
But what we found in in Kenya is that both the kids benefit the low achievement kids and the high achievement kids.

761
01:24:31,900 --> 01:24:40,930
So I'm actually not aware of a similar trial in a in a rich country.

762
01:24:40,930 --> 01:24:45,710
So I don't I don't know the evidence you're referring to,

763
01:24:45,710 --> 01:24:54,520
but it is actually plausible that it could be that the effects are quite different in in rich countries, in particular in countries where,

764
01:24:54,520 --> 01:25:02,470
for example, in the US, where all the public school system is under no child left behind tends to of the opposite set

765
01:25:02,470 --> 01:25:10,390
up where a lot of the attention is is is put to the lower achieving kids from the get go.

766
01:25:10,390 --> 01:25:13,480
So in this in this setting, if you track,

767
01:25:13,480 --> 01:25:24,580
then the kids who are the high achievement kids get suddenly all on the high achievement kids and also a curriculum that's more appropriate to them.

768
01:25:24,580 --> 01:25:31,600
So they will benefit a lot. The low achievement kids that don't benefit from the change in the curriculum because the

769
01:25:31,600 --> 01:25:36,700
curriculum was already appropriate to them and they've lost the high achievement kids,

770
01:25:36,700 --> 01:25:42,550
which then to pull them up anyway, so then they could suffer from it.

771
01:25:42,550 --> 01:25:47,710
So it's the fact that the this is related.

772
01:25:47,710 --> 01:25:51,940
It's kind of when you push the local problems globally. And if you don't, you know,

773
01:25:51,940 --> 01:26:02,320
the fact that the trekking in Kenya and in India benefits the Lord's kids at least as much the high achievement kids in India,

774
01:26:02,320 --> 01:26:09,130
we found the low, high achievement kids who stayed with the teacher to not benefit at all from the programme, but that was not experimental.

775
01:26:09,130 --> 01:26:20,890
But in Kenya, it was experimental. And so the the that reflect the emphasis that the extreme elitism of the curriculum and that may not apply

776
01:26:20,890 --> 01:26:34,260
in in the UK or in the US if the curriculum is more teaching to the median child or teaching to the bottom.

777
01:26:34,260 --> 01:26:46,670
It might make sense to pull questions if they. We just have an indication of who else would like to ask a question on here.

778
01:26:46,670 --> 01:26:51,680
Thank you very much for your hesitation.

779
01:26:51,680 --> 01:27:05,870
It is mentioned the difficulties of scaling up and the need to spread through the system, especially with interventions changing teacher training,

780
01:27:05,870 --> 01:27:14,340
where sometimes it's when you actually go into the initial teaching and education systems, but also the additional challenges.

781
01:27:14,340 --> 01:27:23,960
So whether that's a university system or maybe an untested university programme versus another certification.

782
01:27:23,960 --> 01:27:31,870
This is a different political system of its own with its own.

783
01:27:31,870 --> 01:27:40,240
Priorities, I suppose, but the fact is the possible way of of creating a more sustainable change.

784
01:27:40,240 --> 01:27:48,890
And if there is any push, I don't know where I am sure any of you.

785
01:27:48,890 --> 01:28:06,180
We have a great team of people. The question just up here, I think we're.

786
01:28:06,180 --> 01:28:11,460
OK, bike. Yes, sir, thank you very much. Very interesting presentation.

787
01:28:11,460 --> 01:28:15,990
And it seemed to me that particularly the randomised controlled trial,

788
01:28:15,990 --> 01:28:24,510
that actually all the interventions that you were talking about relied very strongly on.

789
01:28:24,510 --> 01:28:31,050
Essentially, you know, very external form of expertise is imposed from the top on, you know,

790
01:28:31,050 --> 01:28:38,310
very often the community level school education system where you described the parents as being

791
01:28:38,310 --> 01:28:43,980
different and some of the teachers is actively hostile to the units of the intervention.

792
01:28:43,980 --> 01:28:49,320
So I was just wondering if any of your interventions had taken into account.

793
01:28:49,320 --> 01:28:55,260
You were kind of democratising this kind of expertise from, you know, starting off,

794
01:28:55,260 --> 01:29:05,050
trying to get the communities, all the teachers on board before imposing the the child intervention.

795
01:29:05,050 --> 01:29:26,570
Let's take just one more than a few. Thank you very much, very much enjoyed the panel's views.

796
01:29:26,570 --> 01:29:32,200
So Professor Duplo emphasised how schools can be very ineffective in developing country,

797
01:29:32,200 --> 01:29:36,960
especially when it comes to imparting basic knowledge and skills.

798
01:29:36,960 --> 01:29:46,790
But we still find things such as the strong effects of sex education on child mortality or other kinds of outcomes which are very positive.

799
01:29:46,790 --> 01:29:54,650
Also, infertility, for example, we see the least association of relationships between improved schooling and lower fertility.

800
01:29:54,650 --> 01:29:58,130
As well as that he was on interception and so on.

801
01:29:58,130 --> 01:30:06,770
So I wonder how we square these, these these two things, how do we reconcile the fact that we can have positive effects of schooling on some outcomes,

802
01:30:06,770 --> 01:30:16,540
especially democratic outcomes, despite schooling being so poor in these contexts?

803
01:30:16,540 --> 01:30:17,890
So I'm going to start with this one.

804
01:30:17,890 --> 01:30:28,370
So I say 50 percent of kids can can read that stand up to level one down stand up five, not zero and 50 percent is much more than zero.

805
01:30:28,370 --> 01:30:34,670
So the the fact that the school system is doing very poorly doesn't mean that they are doing nothing.

806
01:30:34,670 --> 01:30:42,470
In fact, that's what the Rachel say when you when you bring kids to school for whatever exogenous reason, they learn more.

807
01:30:42,470 --> 01:30:46,010
So in fact, what you said about activity, we also have causal evidence of that.

808
01:30:46,010 --> 01:30:53,840
I've done to try it on secondary school where we give kids a scholarship to go to high schools in Ghana and they went to school, nice school.

809
01:30:53,840 --> 01:30:57,470
We've been following them for 10 years and after 10 years we find a day.

810
01:30:57,470 --> 01:31:01,680
They know many more things, including even practical things,

811
01:31:01,680 --> 01:31:12,070
other than we give them a test of ability to want to analyse data with information on their bucket list.

812
01:31:12,070 --> 01:31:16,190
They are much better at doing all of that than the ones who didn't go.

813
01:31:16,190 --> 01:31:22,340
The point is not that school does not seem to point. Its core does much, much, much, much less than it should be doing.

814
01:31:22,340 --> 01:31:29,390
So the two are entirely squared, and I think it was useful for Rachel to win season on the second question.

815
01:31:29,390 --> 01:31:37,380
Let me take strong exception to what you said. What is imposed from outside is that curriculum.

816
01:31:37,380 --> 01:31:45,410
What the organisation that is pushing teaching at the right level, Peter, it's a huge community movement.

817
01:31:45,410 --> 01:31:55,000
They are the president in every village when, you know, the district education officer, I've never set foot in a place.

818
01:31:55,000 --> 01:32:04,030
You go to a meeting with these guys and they tell you things that are district or even in the in the ministries,

819
01:32:04,030 --> 01:32:12,750
and they are talking about the reality in the villages. In a way that makes it absolutely obvious that they have never gone.

820
01:32:12,750 --> 01:32:17,340
Our community by now, who is the head of Putnam, mentioned some,

821
01:32:17,340 --> 01:32:26,340
some some district in like a remote corner of the upper deck, where she said only Gundy and penalty went there.

822
01:32:26,340 --> 01:32:33,930
So I just like it. So it just so and tool what you just said that I think it's it's worth bearing it.

823
01:32:33,930 --> 01:32:39,270
It is not imposed from the it's not outside expecting. It is Putnam's programme.

824
01:32:39,270 --> 01:32:44,760
Playtime is, you know, extremely would be, let's put it this way.

825
01:32:44,760 --> 01:32:53,310
And in fact, the first time that we evaluated the programme in a rural area, which was with the we reached her,

826
01:32:53,310 --> 01:33:06,330
it was part of the way it was kind of brought in is that there was first the community test testing organised by the community of all the kids so

827
01:33:06,330 --> 01:33:15,420
that they could see that the kids knew much more and it much less than this would because everybody says somewhat over emphasises what kids know.

828
01:33:15,420 --> 01:33:21,120
And then from there on it, the first time was precisely trying to say, What did you sell your options?

829
01:33:21,120 --> 01:33:25,770
You can go and complain to the district education official within the system.

830
01:33:25,770 --> 01:33:33,240
And then the self treatment was like, Well, this is one possibility to send us a kid that the model is not in the school,

831
01:33:33,240 --> 01:33:38,610
not within the school is run by local volunteers. They are not paid.

832
01:33:38,610 --> 01:33:42,720
They are young kids who of you just devote their time to that.

833
01:33:42,720 --> 01:33:49,530
And that's the one that and beautiful. So the issue is certainly not that it's coming from outside.

834
01:33:49,530 --> 01:33:57,270
The issue is that indeed your right to duty us, the teacher actually like just fine.

835
01:33:57,270 --> 01:34:02,130
They go to the training and they think it's very nice and they like the activities, etcetera.

836
01:34:02,130 --> 01:34:09,510
But so we are done just being a kind of qualitative work around the implementation of the scale of that field,

837
01:34:09,510 --> 01:34:15,690
which was really helpful to understand why I feel and it's not because the teacher didn't like it, but it also did was great.

838
01:34:15,690 --> 01:34:23,690
But they were like, I don't have the time. I have to complete that curriculum. That's what's imposed from outside.

839
01:34:23,690 --> 01:34:35,040
So the teachers here would say very much the same thing, and they're facing just a simple and then Richard, we'll take your question on the curve.

840
01:34:35,040 --> 01:34:40,100
So I'm, you know, trying to work through teaching colleges.

841
01:34:40,100 --> 01:34:49,790
I think I mean, we have thought about that. And obviously, you know, if you're trying to change the way people teach, you know,

842
01:34:49,790 --> 01:34:54,230
you might think that a good place to start is teaching teaching colleges.

843
01:34:54,230 --> 01:34:58,670
I think the fundamental problem, though, is what if you do that?

844
01:34:58,670 --> 01:35:08,570
If the incentives within the school system are still to complete the the curriculum, or it might be slightly different in other countries.

845
01:35:08,570 --> 01:35:12,740
So it's not a law to complete the curriculum in many African countries,

846
01:35:12,740 --> 01:35:23,270
but it is nevertheless the incentives are all around getting kids through that final primary school exam or final secondary school exam.

847
01:35:23,270 --> 01:35:32,660
You know, the entire curricula is based around this one final exam, which very, very few people will pass.

848
01:35:32,660 --> 01:35:38,610
And so even if you teach a different method and you still have those incentives,

849
01:35:38,610 --> 01:35:46,430
then then you're going to have problems and you certainly see examples of where you have training and then teachers go

850
01:35:46,430 --> 01:35:52,520
back into the environment and sort of go back to the old ways of teaching unless you fix all the things around it,

851
01:35:52,520 --> 01:35:59,840
which in that curricular example in Kenya that I talked about, it was, you know,

852
01:35:59,840 --> 01:36:07,700
it was pretty comprehensive in that they spend a lot of time understanding where the children were and developing appropriate curricula,

853
01:36:07,700 --> 01:36:15,800
but then went with training the teachers, producing new textbooks, having very guided lessons plan.

854
01:36:15,800 --> 01:36:21,470
So you've got to shift a lot of things if you're going to change the whole system.

855
01:36:21,470 --> 01:36:26,180
And if you just do teaching colleges without the rest, it may not work.

856
01:36:26,180 --> 01:36:34,280
But if if we can move to a new system now, hopefully in Kenya, they're being trained on that new system when they go through teaching colleges.

857
01:36:34,280 --> 01:36:35,660
I think just a general point,

858
01:36:35,660 --> 01:36:44,090
you're just making this a subject of research is going to help you to really try to research whether things work, whether they don't work.

859
01:36:44,090 --> 01:36:47,300
And also, you know, how do you actually get things delivered? And you know what?

860
01:36:47,300 --> 01:36:53,720
What really is still working 10 years after you after you've done just observational studies,

861
01:36:53,720 --> 01:37:01,080
randomised studies, but basically research is going to help balance education.

862
01:37:01,080 --> 01:37:07,730
Easily measurable things are the great progress and of course, the more difficult things to measure, you know, later on in life.

863
01:37:07,730 --> 01:37:11,750
Do people, you know, have very big cost differences? Are they, you know,

864
01:37:11,750 --> 01:37:18,530
the split between grammar schools and secondary boarding schools here became what was mostly abolished because people

865
01:37:18,530 --> 01:37:26,540
didn't like the effects on the English class system rather than on the rather than for any reasons of academic attainment.

866
01:37:26,540 --> 01:37:30,800
And things like that are going to be extremely difficult to assess or virtually impossible.

867
01:37:30,800 --> 01:37:37,340
But at least the things of academic achievement that they are worth assessing should be assessed that are being assessed.

868
01:37:37,340 --> 01:37:45,440
And that's an improvement. Can I can I just come back on the track because I think it's really important when we talk about the role of tracking,

869
01:37:45,440 --> 01:37:50,210
you know, particularly in an English context, when we talk about the role of tracking, it is not about you.

870
01:37:50,210 --> 01:37:58,550
Take one test, you get put in a school and you there for the rest of your life, you know, in the lower level school, which is the English system,

871
01:37:58,550 --> 01:38:08,180
you know, in the most effective programme that Esther was talking about, you get put in it, you know, in different groups, depending on what you know.

872
01:38:08,180 --> 01:38:15,800
And then as soon as you learn that thing, you move up. Right. So it's a very dynamic kind of tracking, which is very different.

873
01:38:15,800 --> 01:38:31,110
So I just, you know, given that whole English history, I think it's important to emphasise that you just take one and there was one person where.

874
01:38:31,110 --> 01:38:38,140
What I would like to go to the first time that you showed us, which is that a biotechnology loads and loads of vegetation,

875
01:38:38,140 --> 01:38:44,800
and I would like to use on the other aspects of education that we did that upon,

876
01:38:44,800 --> 01:38:52,120
which is that all the bands reading want their leads to academy that new suffered in terms of low maintenance knows,

877
01:38:52,120 --> 01:38:59,020
and there's lots of cases of schooling and even bidding by the parents to the children.

878
01:38:59,020 --> 01:39:04,330
If they don't perform well in the tests and exams and ran on a judicial system,

879
01:39:04,330 --> 01:39:12,130
I'll be making this to incorporate those levels of skills in the in the kids,

880
01:39:12,130 --> 01:39:25,010
in the confidence, in terms of happiness, in terms of life skills and not just numeracy and literacy.

881
01:39:25,010 --> 01:39:33,650
So in our pre-school programme, the active control, if you remember, was a social skill was a social skill intervention.

882
01:39:33,650 --> 01:39:43,130
And the the reason why we picked a social skill is not only we felt that probably that it was a good,

883
01:39:43,130 --> 01:39:50,210
good, good way to have the same level of games and stuff like that, but not the mass content.

884
01:39:50,210 --> 01:39:53,960
So it was a good comparison, but also because we could feel that we could.

885
01:39:53,960 --> 01:40:01,680
We felt confident to say with a straight face that actually this is going to be helpful for the kids.

886
01:40:01,680 --> 01:40:03,800
We hope it's going to be helpful for other kids.

887
01:40:03,800 --> 01:40:15,530
And in fact, we did find an effective social skills training on social skills, and there is a lot of emphasis on that on score,

888
01:40:15,530 --> 01:40:29,720
as does the teaching life skills more in the in in the US and in other rich countries, in particular on the world of the work of Carol Dweck.

889
01:40:29,720 --> 01:40:33,080
Thinking about the mindset,

890
01:40:33,080 --> 01:40:44,390
the importance of of seeing the world as a place that you can change and that to your own point as something that can change,

891
01:40:44,390 --> 01:40:52,790
which has Bozer effect on the on happiness and effect on academic skills.

892
01:40:52,790 --> 01:41:01,970
But I think you are absolutely right to say that it's one of the thing that we have in a lot of developing countries education context.

893
01:41:01,970 --> 01:41:09,680
We are light years away from, from incorporating these things.

894
01:41:09,680 --> 01:41:13,460
And I think you are right to mention the whole of the of the balance in there,

895
01:41:13,460 --> 01:41:23,750
which is to some extent it does reflect that aren't ambitious for their children, which is perfectly understandable and better than the opposite.

896
01:41:23,750 --> 01:41:28,220
But where they constantly do vision, it can be counterproductive.

897
01:41:28,220 --> 01:41:32,600
And another thing that we didn't touch upon is that when one of the things that is pretty

898
01:41:32,600 --> 01:41:39,080
heartbreaking in that is that that also lead to a lot of divergence in the education performance,

899
01:41:39,080 --> 01:41:40,940
even within the same family,

900
01:41:40,940 --> 01:41:50,870
where on the balance of identifying who they think is the bright one and then putting all of their education and eggs into that one basket.

901
01:41:50,870 --> 01:41:55,760
So, for example, cricket in the family, there is one bright one, by the way.

902
01:41:55,760 --> 01:41:57,560
I should said it, it's not necessarily the boy.

903
01:41:57,560 --> 01:42:02,690
It could be that they decide that it's the girl who is the right one, even in India, where there is always, always.

904
01:42:02,690 --> 01:42:07,370
But then to the girl by boy or girl is being sent to private school.

905
01:42:07,370 --> 01:42:17,480
Everybody else contributes to that. In an off, it said if you could that create pressure both on that side and on the other kids,

906
01:42:17,480 --> 01:42:25,820
another thing that is a little bit disturbing in that is that there is a very nice paper from

907
01:42:25,820 --> 01:42:34,140
Malawi which shows that parents do that on the basis of very shaky information about again,

908
01:42:34,140 --> 01:42:35,570
I'm not even talking about ability,

909
01:42:35,570 --> 01:42:42,830
but even the achievement of their children because the information is not given to them in the most transparent of ways.

910
01:42:42,830 --> 01:42:50,810
So they form an opinion, which is their only somewhat correlated with how well the kids are actually doing.

911
01:42:50,810 --> 01:42:56,180
So it's a work by if I had journals that Chicago's was just trying to show that she's doing that.

912
01:42:56,180 --> 01:43:04,130
The parents are trying to get their day of their investment to what they think is the achievement of the child.

913
01:43:04,130 --> 01:43:10,340
But they're failing because they don't really know what the achievement of the charities and when you give them information on that just grade.

914
01:43:10,340 --> 01:43:16,610
So nothing gets it, but just grade the they the response to it is very, very steep.

915
01:43:16,610 --> 01:43:26,150
So that's kind of adds a layer to that, which is the heating and the pressure is not even necessarily based on very, very strong evidence.

916
01:43:26,150 --> 01:43:38,720
And I think it does. I did touched upon that rapidly, but it does contribute to the to the failure of the system in the sense that every boys,

917
01:43:38,720 --> 01:43:47,490
the band, the children and the teacher all internalise whatever happens to be the ranking at some point and we fight.

918
01:43:47,490 --> 01:43:55,970
That's why I kind of reacted into achievement vs. ability because they make it like a thing which is immutable,

919
01:43:55,970 --> 01:44:06,500
even though it could just be like, you know, 10 kids age and whatnot, and they would have been able to to find a fit again.

920
01:44:06,500 --> 01:44:13,820
But they don't get that sense because their fell down fell down when they finally persuaded themselves that it's not for them.

921
01:44:13,820 --> 01:44:25,740
And and that's why you have kids who can calculate at the speed of light who tell you that they cannot do the subtraction.

922
01:44:25,740 --> 01:44:30,950
And that's it, that's the tragedy. And I'm completely with you on that.

923
01:44:30,950 --> 01:44:37,560
And I think we should draw this to a close now, and I'd like to invite everyone here to join us.

924
01:44:37,560 --> 01:44:43,680
The reception, which is in the area outside the lecture theatre here, and there'll be an opportunity,

925
01:44:43,680 --> 01:44:48,000
maybe two, to ask further questions of the panellists if you have them.

926
01:44:48,000 --> 01:44:55,800
We started with a very big question and I'd like to thank all three panellists for actually rising to the challenge and helping us to answer it.

927
01:44:55,800 --> 01:45:07,654
So thank you.